Skip to main content

Bash script and characters that require to be escaped

There are few ASCII characters that need to be escaped when using in certain bash scripts especially when echoing or passing value through variables. What are those characters can be determined using the small bash script below.

#!/bin/bash
special=$'0123456789\'-–—!"#$%&()*,./:;?@[\]^_`{|}~¡¦¨¯´¸¿‘’‚“”„¢£¤¥€+<=>±«»×÷§©¬®°µ¶·…†‡•‰¼½¾¹²³AaªÁáÀàÂâÄäÃãÅåÆæBbCcÇçDdÐðEeÉéÈèÊêËëFfƒGgHhIiÍíÌìÎîÏïJjKkLlMmNnÑñOoºÓóÒòÔôÖöÕõØøŒœPpQqRrSsŠšßTtÞþ™UuÚúÙùÛûÜüVvWwXxYyÝýÿŸZz'
for ((i=0; i < ${#special}; i++)); do
    char="${special:i:1}"
    printf -v q_char '%q' "$char"
    if [[ "$char" != "$q_char" ]]; then
        printf 'Yes - character %s needs to be escaped\n' "$char"
    else
        printf 'No - character %s does not need to be escaped\n' "$char"
    fi
done | sort

 The output of above script will list characters that need escaping and looks like below.

 [me@mysystem]$ ./esc.sh
No - character 0 does not need to be escaped
No - character 1 does not need to be escaped
No - character 2 does not need to be escaped
No - character 3 does not need to be escaped
No - character 4 does not need to be escaped
No - character 5 does not need to be escaped
No - character 6 does not need to be escaped
No - character 7 does not need to be escaped
No - character 8 does not need to be escaped
No - character 9 does not need to be escaped
No - character a does not need to be escaped
No - character A does not need to be escaped
No - character b does not need to be escaped
No - character B does not need to be escaped
No - character c does not need to be escaped
No - character C does not need to be escaped
No - character d does not need to be escaped
No - character D does not need to be escaped
No - character ~ does not need to be escaped
No - character = does not need to be escaped
No - character _ does not need to be escaped
No - character - does not need to be escaped
No - character : does not need to be escaped
No - character / does not need to be escaped
No - character . does not need to be escaped
No - character @ does not need to be escaped
No - character % does not need to be escaped
No - character + does not need to be escaped
No - character e does not need to be escaped
No - character E does not need to be escaped
No - character f does not need to be escaped
No - character F does not need to be escaped
No - character g does not need to be escaped
No - character G does not need to be escaped
No - character h does not need to be escaped
No - character H does not need to be escaped
No - character i does not need to be escaped
No - character I does not need to be escaped
No - character j does not need to be escaped
No - character J does not need to be escaped
No - character k does not need to be escaped
No - character K does not need to be escaped
No - character l does not need to be escaped
No - character L does not need to be escaped
No - character m does not need to be escaped
No - character M does not need to be escaped
No - character n does not need to be escaped
No - character N does not need to be escaped
No - character o does not need to be escaped
No - character O does not need to be escaped
No - character p does not need to be escaped
No - character P does not need to be escaped
No - character q does not need to be escaped
No - character Q does not need to be escaped
No - character r does not need to be escaped
No - character R does not need to be escaped
No - character s does not need to be escaped
No - character S does not need to be escaped
No - character t does not need to be escaped
No - character T does not need to be escaped
No - character u does not need to be escaped
No - character U does not need to be escaped
No - character v does not need to be escaped
No - character V does not need to be escaped
No - character w does not need to be escaped
No - character W does not need to be escaped
No - character x does not need to be escaped
No - character X does not need to be escaped
No - character y does not need to be escaped
No - character Y does not need to be escaped
No - character z does not need to be escaped
No - character Z does not need to be escaped
Yes - character ¼ needs to be escaped
Yes - character ½ needs to be escaped
Yes - character ¾ needs to be escaped
Yes - character ¹ needs to be escaped
Yes - character ² needs to be escaped
Yes - character ³ needs to be escaped
Yes - character æ needs to be escaped
Yes - character Æ needs to be escaped
Yes - character ª needs to be escaped
Yes - character á needs to be escaped
Yes - character Á needs to be escaped
Yes - character à needs to be escaped
Yes - character À needs to be escaped
Yes - character â needs to be escaped
Yes - character  needs to be escaped
Yes - character å needs to be escaped
Yes - character Å needs to be escaped
Yes - character ä needs to be escaped
Yes - character Ä needs to be escaped
Yes - character ã needs to be escaped
Yes - character à needs to be escaped
Yes - character ç needs to be escaped
Yes - character Ç needs to be escaped
Yes - character ð needs to be escaped
Yes - character Ð needs to be escaped
Yes - character é needs to be escaped
Yes - character É needs to be escaped
Yes - character è needs to be escaped
Yes - character È needs to be escaped
Yes - character ê needs to be escaped
Yes - character Ê needs to be escaped
Yes - character ë needs to be escaped
Yes - character Ë needs to be escaped
Yes - character í needs to be escaped
Yes - character Í needs to be escaped
Yes - character ì needs to be escaped
Yes - character Ì needs to be escaped
Yes - character î needs to be escaped
Yes - character Î needs to be escaped
Yes - character ï needs to be escaped
Yes - character Ï needs to be escaped
Yes - character ´ needs to be escaped
Yes - character ` needs to be escaped
Yes - character ^ needs to be escaped
Yes - character ¨ needs to be escaped
Yes - character ÷ needs to be escaped
Yes - character × needs to be escaped
Yes - character < needs to be escaped
Yes - character > needs to be escaped
Yes - character ¬ needs to be escaped
Yes - character | needs to be escaped
Yes - character ¦ needs to be escaped
Yes - character ° needs to be escaped
Yes - character µ needs to be escaped
Yes - character ¯ needs to be escaped
Yes - character , needs to be escaped
Yes - character ; needs to be escaped
Yes - character ! needs to be escaped
Yes - character ¡ needs to be escaped
Yes - character ? needs to be escaped
Yes - character ¿ needs to be escaped
Yes - character · needs to be escaped
Yes - character ¸ needs to be escaped
Yes - character ' needs to be escaped
Yes - character ‘ needs to be escaped
Yes - character ’ needs to be escaped
Yes - character " needs to be escaped
Yes - character “ needs to be escaped
Yes - character ” needs to be escaped
Yes - character « needs to be escaped
Yes - character » needs to be escaped
Yes - character ( needs to be escaped
Yes - character ) needs to be escaped
Yes - character [ needs to be escaped
Yes - character ] needs to be escaped
Yes - character { needs to be escaped
Yes - character } needs to be escaped
Yes - character § needs to be escaped
Yes - character ¶ needs to be escaped
Yes - character © needs to be escaped
Yes - character ® needs to be escaped
Yes - character ™ needs to be escaped
Yes - character ¤ needs to be escaped
Yes - character ¢ needs to be escaped
Yes - character $ needs to be escaped
Yes - character £ needs to be escaped
Yes - character ¥ needs to be escaped
Yes - character € needs to be escaped
Yes - character * needs to be escaped
Yes - character \ needs to be escaped
Yes - character & needs to be escaped
Yes - character # needs to be escaped
Yes - character ± needs to be escaped
Yes - character ƒ needs to be escaped
Yes - character – needs to be escaped
Yes - character — needs to be escaped
Yes - character ‚ needs to be escaped
Yes - character „ needs to be escaped
Yes - character … needs to be escaped
Yes - character † needs to be escaped
Yes - character ‡ needs to be escaped
Yes - character • needs to be escaped
Yes - character ‰ needs to be escaped
Yes - character ñ needs to be escaped
Yes - character Ñ needs to be escaped
Yes - character œ needs to be escaped
Yes - character Πneeds to be escaped
Yes - character º needs to be escaped
Yes - character ó needs to be escaped
Yes - character Ó needs to be escaped
Yes - character ò needs to be escaped
Yes - character Ò needs to be escaped
Yes - character ô needs to be escaped
Yes - character Ô needs to be escaped
Yes - character ö needs to be escaped
Yes - character Ö needs to be escaped
Yes - character õ needs to be escaped
Yes - character Õ needs to be escaped
Yes - character ø needs to be escaped
Yes - character Ø needs to be escaped
Yes - character š needs to be escaped
Yes - character Š needs to be escaped
Yes - character ß needs to be escaped
Yes - character ú needs to be escaped
Yes - character Ú needs to be escaped
Yes - character ù needs to be escaped
Yes - character Ù needs to be escaped
Yes - character û needs to be escaped
Yes - character Û needs to be escaped
Yes - character ü needs to be escaped
Yes - character Ü needs to be escaped
Yes - character ý needs to be escaped
Yes - character Ý needs to be escaped
Yes - character ÿ needs to be escaped
Yes - character Ÿ needs to be escaped
Yes - character þ needs to be escaped
Yes - character Þ needs to be escaped
[me@mysystem]$

So how do you use the escape character?

 Simply change the character either to the ASCII code or add a backslash before the character, e.g. you will change $ to \$.

 Another way to list all the ASCII characters could be achieved using script below.

 for i in {0..127} ;do
    printf -v var \\%o $i
    printf -v var $var
    printf -v res "%q" "$var"
    esc=Escape
    [ "$var" = "$res" ] && esc=NoEsc
    printf "%02X %s %-7s\n" $i $esc "$res"
done |

    column

The output will then look like below.

 [me@mysystem]$ ./esc2.sh
00 Escape ''            20 Escape \             40 NoEsc @              60 Escape \`
01 Escape $'\001'       21 Escape \!            41 NoEsc A              61 NoEsc a
02 Escape $'\002'       22 Escape \"            42 NoEsc B              62 NoEsc b
03 Escape $'\003'       23 Escape \#            43 NoEsc C              63 NoEsc c
04 Escape $'\004'       24 Escape \$            44 NoEsc D              64 NoEsc d
05 Escape $'\005'       25 NoEsc %              45 NoEsc E              65 NoEsc e
06 Escape $'\006'       26 Escape \&            46 NoEsc F              66 NoEsc f
07 Escape $'\a'         27 Escape \'            47 NoEsc G              67 NoEsc g
08 Escape $'\b'         28 Escape \(            48 NoEsc H              68 NoEsc h
09 Escape $'\t'         29 Escape \)            49 NoEsc I              69 NoEsc i
0A Escape $'\n'         2A Escape \*            4A NoEsc J              6A NoEsc j
0B Escape $'\v'         2B NoEsc +              4B NoEsc K              6B NoEsc k
0C Escape $'\f'         2C Escape \,            4C NoEsc L              6C NoEsc l
0D Escape $'\r'         2D NoEsc -              4D NoEsc M              6D NoEsc m
0E Escape $'\016'       2E NoEsc .              4E NoEsc N              6E NoEsc n
0F Escape $'\017'       2F NoEsc /              4F NoEsc O              6F NoEsc o
10 Escape $'\020'       30 NoEsc 0              50 NoEsc P              70 NoEsc p
11 Escape $'\021'       31 NoEsc 1              51 NoEsc Q              71 NoEsc q
12 Escape $'\022'       32 NoEsc 2              52 NoEsc R              72 NoEsc r
13 Escape $'\023'       33 NoEsc 3              53 NoEsc S              73 NoEsc s
14 Escape $'\024'       34 NoEsc 4              54 NoEsc T              74 NoEsc t
15 Escape $'\025'       35 NoEsc 5              55 NoEsc U              75 NoEsc u
16 Escape $'\026'       36 NoEsc 6              56 NoEsc V              76 NoEsc v
17 Escape $'\027'       37 NoEsc 7              57 NoEsc W              77 NoEsc w
18 Escape $'\030'       38 NoEsc 8              58 NoEsc X              78 NoEsc x
19 Escape $'\031'       39 NoEsc 9              59 NoEsc Y              79 NoEsc y
1A Escape $'\032'       3A NoEsc :              5A NoEsc Z              7A NoEsc z
1B Escape $'\E'         3B Escape \;            5B Escape \[            7B Escape \{
1C Escape $'\034'       3C Escape \<            5C Escape \\            7C Escape \|
1D Escape $'\035'       3D NoEsc =              5D Escape \]            7D Escape \}
1E Escape $'\036'       3E Escape \>            5E Escape \^            7E NoEsc ~
1F Escape $'\037'       3F Escape \?            5F NoEsc _              7F Escape $'\177'

[me@mysystem]$

Popular posts from this blog

Useful website performance and load testing tools

http://tsung.erlang-projects.org/ http://httpd.apache.org/docs/2.0/programs/ab.html http://phantomjs.org/ https://developers.google.com/speed/pagespeed/ http://servermonitoringhq.com/blog/how_to_quickly_stress_test_a_web_server https://code.google.com/p/httperf/ http://loadimpact.com/ http://www.paessler.com/webstress http://loaduiweb.org/ http://en.wikipedia.org/wiki/Web_server_benchmarking http://en.wikipedia.org/wiki/Load_testing http://www.loadui.org/ http://www.loadtestingtool.com/index.shtml http://www.appdynamics.com/blog/devops/load-testing-tools-explained-the-server-side/
Read more

Copy files and folders using SCP with spaces in path

Copying data from one system to other with file or folder names that contain spaces in path can be achieved using this guide. In this case I am copying data from Macbook to Windows 10 computer. In order to copy the data easily it is better to use bash commands. Windows computer can support WSL (Windows subsystem for Linux) and you can run one of few linux distributions to use shell commands. I have Ubuntu set up within my Windows 10 using WSL. If you do not have WSL, you can set it up using my guide here . The copy can be performed in two ways: 1) Using SCP Source (MacOs) path: /home/Users/username/Documents/data extract from 2020/ First of all you add escape sequence to the path so it will become:  /home/Users/me/Documents/data\ extract\ from\ 2020/ . While this works on local system for SCP you'll have to double the escape sequences by replacing \ with \\, as below. Figure out your source computer IP address using "ifconfig" command. Now using scp command on target syst...
Read more

TrueCrypt on macOS X Mojave 10.14

If you have updated your macOS recently to Mojave otherwise known as verison 10.14 you may not be able to install the last version of Truecrypt in order to access your old volumes encrypted with Truecrypt software. This article will guide you to get this working on your MacOS v10.14 (Mjoave) . Download the package from  https://truecrypt.ch/downloads/  or  https://www.truecrypt71a.com/downloads/ . Find downloaded package using Finder in your HDD/Users/username/Downloads folder and will look like  TrueCrypt 7.1a Mac OS X.dmg . Open file location in Finder and open or double click on  TrueCrypt 7.1a Mac OS X.dmg . This will mount Truecrypt 7.1a and will have Truecrypt 7.1a.mpkg in it. Drag the package T rueCrypt 7.1a.mpkg and drop in your Downloads folder. From Locations in Finder you can eject your TrueCrypt mount. Now go to your Downloads location, find the file  TrueCrypt 7.1a.mpkg , right click and select Show Package Contents . Find the ...
Read more