Skip to main content

Bash script and characters that require to be escaped

There are few ASCII characters that need to be escaped when using in certain bash scripts especially when echoing or passing value through variables. What are those characters can be determined using the small bash script below.

#!/bin/bash
special=$'0123456789\'-–—!"#$%&()*,./:;?@[\]^_`{|}~¡¦¨¯´¸¿‘’‚“”„¢£¤¥€+<=>±«»×÷§©¬®°µ¶·…†‡•‰¼½¾¹²³AaªÁáÀàÂâÄäÃãÅåÆæBbCcÇçDdÐðEeÉéÈèÊêËëFfƒGgHhIiÍíÌìÎîÏïJjKkLlMmNnÑñOoºÓóÒòÔôÖöÕõØøŒœPpQqRrSsŠšßTtÞþ™UuÚúÙùÛûÜüVvWwXxYyÝýÿŸZz'
for ((i=0; i < ${#special}; i++)); do
    char="${special:i:1}"
    printf -v q_char '%q' "$char"
    if [[ "$char" != "$q_char" ]]; then
        printf 'Yes - character %s needs to be escaped\n' "$char"
    else
        printf 'No - character %s does not need to be escaped\n' "$char"
    fi
done | sort

 The output of above script will list characters that need escaping and looks like below.

 [me@mysystem]$ ./esc.sh
No - character 0 does not need to be escaped
No - character 1 does not need to be escaped
No - character 2 does not need to be escaped
No - character 3 does not need to be escaped
No - character 4 does not need to be escaped
No - character 5 does not need to be escaped
No - character 6 does not need to be escaped
No - character 7 does not need to be escaped
No - character 8 does not need to be escaped
No - character 9 does not need to be escaped
No - character a does not need to be escaped
No - character A does not need to be escaped
No - character b does not need to be escaped
No - character B does not need to be escaped
No - character c does not need to be escaped
No - character C does not need to be escaped
No - character d does not need to be escaped
No - character D does not need to be escaped
No - character ~ does not need to be escaped
No - character = does not need to be escaped
No - character _ does not need to be escaped
No - character - does not need to be escaped
No - character : does not need to be escaped
No - character / does not need to be escaped
No - character . does not need to be escaped
No - character @ does not need to be escaped
No - character % does not need to be escaped
No - character + does not need to be escaped
No - character e does not need to be escaped
No - character E does not need to be escaped
No - character f does not need to be escaped
No - character F does not need to be escaped
No - character g does not need to be escaped
No - character G does not need to be escaped
No - character h does not need to be escaped
No - character H does not need to be escaped
No - character i does not need to be escaped
No - character I does not need to be escaped
No - character j does not need to be escaped
No - character J does not need to be escaped
No - character k does not need to be escaped
No - character K does not need to be escaped
No - character l does not need to be escaped
No - character L does not need to be escaped
No - character m does not need to be escaped
No - character M does not need to be escaped
No - character n does not need to be escaped
No - character N does not need to be escaped
No - character o does not need to be escaped
No - character O does not need to be escaped
No - character p does not need to be escaped
No - character P does not need to be escaped
No - character q does not need to be escaped
No - character Q does not need to be escaped
No - character r does not need to be escaped
No - character R does not need to be escaped
No - character s does not need to be escaped
No - character S does not need to be escaped
No - character t does not need to be escaped
No - character T does not need to be escaped
No - character u does not need to be escaped
No - character U does not need to be escaped
No - character v does not need to be escaped
No - character V does not need to be escaped
No - character w does not need to be escaped
No - character W does not need to be escaped
No - character x does not need to be escaped
No - character X does not need to be escaped
No - character y does not need to be escaped
No - character Y does not need to be escaped
No - character z does not need to be escaped
No - character Z does not need to be escaped
Yes - character ¼ needs to be escaped
Yes - character ½ needs to be escaped
Yes - character ¾ needs to be escaped
Yes - character ¹ needs to be escaped
Yes - character ² needs to be escaped
Yes - character ³ needs to be escaped
Yes - character æ needs to be escaped
Yes - character Æ needs to be escaped
Yes - character ª needs to be escaped
Yes - character á needs to be escaped
Yes - character Á needs to be escaped
Yes - character à needs to be escaped
Yes - character À needs to be escaped
Yes - character â needs to be escaped
Yes - character  needs to be escaped
Yes - character å needs to be escaped
Yes - character Å needs to be escaped
Yes - character ä needs to be escaped
Yes - character Ä needs to be escaped
Yes - character ã needs to be escaped
Yes - character à needs to be escaped
Yes - character ç needs to be escaped
Yes - character Ç needs to be escaped
Yes - character ð needs to be escaped
Yes - character Ð needs to be escaped
Yes - character é needs to be escaped
Yes - character É needs to be escaped
Yes - character è needs to be escaped
Yes - character È needs to be escaped
Yes - character ê needs to be escaped
Yes - character Ê needs to be escaped
Yes - character ë needs to be escaped
Yes - character Ë needs to be escaped
Yes - character í needs to be escaped
Yes - character Í needs to be escaped
Yes - character ì needs to be escaped
Yes - character Ì needs to be escaped
Yes - character î needs to be escaped
Yes - character Î needs to be escaped
Yes - character ï needs to be escaped
Yes - character Ï needs to be escaped
Yes - character ´ needs to be escaped
Yes - character ` needs to be escaped
Yes - character ^ needs to be escaped
Yes - character ¨ needs to be escaped
Yes - character ÷ needs to be escaped
Yes - character × needs to be escaped
Yes - character < needs to be escaped
Yes - character > needs to be escaped
Yes - character ¬ needs to be escaped
Yes - character | needs to be escaped
Yes - character ¦ needs to be escaped
Yes - character ° needs to be escaped
Yes - character µ needs to be escaped
Yes - character ¯ needs to be escaped
Yes - character , needs to be escaped
Yes - character ; needs to be escaped
Yes - character ! needs to be escaped
Yes - character ¡ needs to be escaped
Yes - character ? needs to be escaped
Yes - character ¿ needs to be escaped
Yes - character · needs to be escaped
Yes - character ¸ needs to be escaped
Yes - character ' needs to be escaped
Yes - character ‘ needs to be escaped
Yes - character ’ needs to be escaped
Yes - character " needs to be escaped
Yes - character “ needs to be escaped
Yes - character ” needs to be escaped
Yes - character « needs to be escaped
Yes - character » needs to be escaped
Yes - character ( needs to be escaped
Yes - character ) needs to be escaped
Yes - character [ needs to be escaped
Yes - character ] needs to be escaped
Yes - character { needs to be escaped
Yes - character } needs to be escaped
Yes - character § needs to be escaped
Yes - character ¶ needs to be escaped
Yes - character © needs to be escaped
Yes - character ® needs to be escaped
Yes - character ™ needs to be escaped
Yes - character ¤ needs to be escaped
Yes - character ¢ needs to be escaped
Yes - character $ needs to be escaped
Yes - character £ needs to be escaped
Yes - character ¥ needs to be escaped
Yes - character € needs to be escaped
Yes - character * needs to be escaped
Yes - character \ needs to be escaped
Yes - character & needs to be escaped
Yes - character # needs to be escaped
Yes - character ± needs to be escaped
Yes - character ƒ needs to be escaped
Yes - character – needs to be escaped
Yes - character — needs to be escaped
Yes - character ‚ needs to be escaped
Yes - character „ needs to be escaped
Yes - character … needs to be escaped
Yes - character † needs to be escaped
Yes - character ‡ needs to be escaped
Yes - character • needs to be escaped
Yes - character ‰ needs to be escaped
Yes - character ñ needs to be escaped
Yes - character Ñ needs to be escaped
Yes - character œ needs to be escaped
Yes - character Πneeds to be escaped
Yes - character º needs to be escaped
Yes - character ó needs to be escaped
Yes - character Ó needs to be escaped
Yes - character ò needs to be escaped
Yes - character Ò needs to be escaped
Yes - character ô needs to be escaped
Yes - character Ô needs to be escaped
Yes - character ö needs to be escaped
Yes - character Ö needs to be escaped
Yes - character õ needs to be escaped
Yes - character Õ needs to be escaped
Yes - character ø needs to be escaped
Yes - character Ø needs to be escaped
Yes - character š needs to be escaped
Yes - character Š needs to be escaped
Yes - character ß needs to be escaped
Yes - character ú needs to be escaped
Yes - character Ú needs to be escaped
Yes - character ù needs to be escaped
Yes - character Ù needs to be escaped
Yes - character û needs to be escaped
Yes - character Û needs to be escaped
Yes - character ü needs to be escaped
Yes - character Ü needs to be escaped
Yes - character ý needs to be escaped
Yes - character Ý needs to be escaped
Yes - character ÿ needs to be escaped
Yes - character Ÿ needs to be escaped
Yes - character þ needs to be escaped
Yes - character Þ needs to be escaped
[me@mysystem]$

So how do you use the escape character?

 Simply change the character either to the ASCII code or add a backslash before the character, e.g. you will change $ to \$.

 Another way to list all the ASCII characters could be achieved using script below.

 for i in {0..127} ;do
    printf -v var \\%o $i
    printf -v var $var
    printf -v res "%q" "$var"
    esc=Escape
    [ "$var" = "$res" ] && esc=NoEsc
    printf "%02X %s %-7s\n" $i $esc "$res"
done |

    column

The output will then look like below.

 [me@mysystem]$ ./esc2.sh
00 Escape ''            20 Escape \             40 NoEsc @              60 Escape \`
01 Escape $'\001'       21 Escape \!            41 NoEsc A              61 NoEsc a
02 Escape $'\002'       22 Escape \"            42 NoEsc B              62 NoEsc b
03 Escape $'\003'       23 Escape \#            43 NoEsc C              63 NoEsc c
04 Escape $'\004'       24 Escape \$            44 NoEsc D              64 NoEsc d
05 Escape $'\005'       25 NoEsc %              45 NoEsc E              65 NoEsc e
06 Escape $'\006'       26 Escape \&            46 NoEsc F              66 NoEsc f
07 Escape $'\a'         27 Escape \'            47 NoEsc G              67 NoEsc g
08 Escape $'\b'         28 Escape \(            48 NoEsc H              68 NoEsc h
09 Escape $'\t'         29 Escape \)            49 NoEsc I              69 NoEsc i
0A Escape $'\n'         2A Escape \*            4A NoEsc J              6A NoEsc j
0B Escape $'\v'         2B NoEsc +              4B NoEsc K              6B NoEsc k
0C Escape $'\f'         2C Escape \,            4C NoEsc L              6C NoEsc l
0D Escape $'\r'         2D NoEsc -              4D NoEsc M              6D NoEsc m
0E Escape $'\016'       2E NoEsc .              4E NoEsc N              6E NoEsc n
0F Escape $'\017'       2F NoEsc /              4F NoEsc O              6F NoEsc o
10 Escape $'\020'       30 NoEsc 0              50 NoEsc P              70 NoEsc p
11 Escape $'\021'       31 NoEsc 1              51 NoEsc Q              71 NoEsc q
12 Escape $'\022'       32 NoEsc 2              52 NoEsc R              72 NoEsc r
13 Escape $'\023'       33 NoEsc 3              53 NoEsc S              73 NoEsc s
14 Escape $'\024'       34 NoEsc 4              54 NoEsc T              74 NoEsc t
15 Escape $'\025'       35 NoEsc 5              55 NoEsc U              75 NoEsc u
16 Escape $'\026'       36 NoEsc 6              56 NoEsc V              76 NoEsc v
17 Escape $'\027'       37 NoEsc 7              57 NoEsc W              77 NoEsc w
18 Escape $'\030'       38 NoEsc 8              58 NoEsc X              78 NoEsc x
19 Escape $'\031'       39 NoEsc 9              59 NoEsc Y              79 NoEsc y
1A Escape $'\032'       3A NoEsc :              5A NoEsc Z              7A NoEsc z
1B Escape $'\E'         3B Escape \;            5B Escape \[            7B Escape \{
1C Escape $'\034'       3C Escape \<            5C Escape \\            7C Escape \|
1D Escape $'\035'       3D NoEsc =              5D Escape \]            7D Escape \}
1E Escape $'\036'       3E Escape \>            5E Escape \^            7E NoEsc ~
1F Escape $'\037'       3F Escape \?            5F NoEsc _              7F Escape $'\177'

[me@mysystem]$

Popular posts from this blog

Useful website performance and load testing tools

http://tsung.erlang-projects.org/ http://httpd.apache.org/docs/2.0/programs/ab.html http://phantomjs.org/ https://developers.google.com/speed/pagespeed/ http://servermonitoringhq.com/blog/how_to_quickly_stress_test_a_web_server https://code.google.com/p/httperf/ http://loadimpact.com/ http://www.paessler.com/webstress http://loaduiweb.org/ http://en.wikipedia.org/wiki/Web_server_benchmarking http://en.wikipedia.org/wiki/Load_testing http://www.loadui.org/ http://www.loadtestingtool.com/index.shtml http://www.appdynamics.com/blog/devops/load-testing-tools-explained-the-server-side/
Read more

VMWare ESXi 6.5 HP Custom Image Upgrade to v6.5U3

I had been escaping the Unhandled Exception error every time I log into my ESXi standalone server running v6.5 on N54L and can't be upgraded to v6.7 due to process compatibility. I have finally found the HP Custom image updated to v6.5U3 which appeared to have the fix built into it. It was quite straightforward update using the l latest 6.5 HP image so as habit I am making notes here for myself and everyone else who may find it useful. First of all I downloaded HP Custom v6.5U3 image  by selecting Offline Bundle. I then placed it in my datastore e.g. ds001. Once copied I ran following command line after I was connected to ESXi using ssh. # esxcli software vib update -d /vmfs/volumes/ds001/VMware-ESXi-6.5.0-Update3-14990892-HPE-preGen9-650.U3.9.6.10.1-Dec2019-depot.zip This command took few seconds or may be minutes but confirmed that updates have been installed and will take effect after reboot. So I rebooted the host and it worked like magic. # reboot I did not place host in main...
Read more

TrueCrypt on macOS X Mojave 10.14

If you have updated your macOS recently to Mojave otherwise known as verison 10.14 you may not be able to install the last version of Truecrypt in order to access your old volumes encrypted with Truecrypt software. This article will guide you to get this working on your MacOS v10.14 (Mjoave) . Download the package from  https://truecrypt.ch/downloads/  or  https://www.truecrypt71a.com/downloads/ . Find downloaded package using Finder in your HDD/Users/username/Downloads folder and will look like  TrueCrypt 7.1a Mac OS X.dmg . Open file location in Finder and open or double click on  TrueCrypt 7.1a Mac OS X.dmg . This will mount Truecrypt 7.1a and will have Truecrypt 7.1a.mpkg in it. Drag the package T rueCrypt 7.1a.mpkg and drop in your Downloads folder. From Locations in Finder you can eject your TrueCrypt mount. Now go to your Downloads location, find the file  TrueCrypt 7.1a.mpkg , right click and select Show Package Contents . Find the ...
Read more