Skip to main content

Bash script and characters that require to be escaped

There are few ASCII characters that need to be escaped when using in certain bash scripts especially when echoing or passing value through variables. What are those characters can be determined using the small bash script below.

#!/bin/bash
special=$'0123456789\'-–—!"#$%&()*,./:;?@[\]^_`{|}~¡¦¨¯´¸¿‘’‚“”„¢£¤¥€+<=>±«»×÷§©¬®°µ¶·…†‡•‰¼½¾¹²³AaªÁáÀàÂâÄäÃãÅåÆæBbCcÇçDdÐðEeÉéÈèÊêËëFfƒGgHhIiÍíÌìÎîÏïJjKkLlMmNnÑñOoºÓóÒòÔôÖöÕõØøŒœPpQqRrSsŠšßTtÞþ™UuÚúÙùÛûÜüVvWwXxYyÝýÿŸZz'
for ((i=0; i < ${#special}; i++)); do
    char="${special:i:1}"
    printf -v q_char '%q' "$char"
    if [[ "$char" != "$q_char" ]]; then
        printf 'Yes - character %s needs to be escaped\n' "$char"
    else
        printf 'No - character %s does not need to be escaped\n' "$char"
    fi
done | sort

The output of above script will list characters that need escaping and looks like below.

[me@mysystem]$ ./esc.sh
No - character 0 does not need to be escaped
No - character 1 does not need to be escaped
No - character 2 does not need to be escaped
No - character 3 does not need to be escaped
No - character 4 does not need to be escaped
No - character 5 does not need to be escaped
No - character 6 does not need to be escaped
No - character 7 does not need to be escaped
No - character 8 does not need to be escaped
No - character 9 does not need to be escaped
No - character a does not need to be escaped
No - character A does not need to be escaped
No - character b does not need to be escaped
No - character B does not need to be escaped
No - character c does not need to be escaped
No - character C does not need to be escaped
No - character d does not need to be escaped
No - character D does not need to be escaped
No - character ~ does not need to be escaped
No - character = does not need to be escaped
No - character _ does not need to be escaped
No - character - does not need to be escaped
No - character : does not need to be escaped
No - character / does not need to be escaped
No - character . does not need to be escaped
No - character @ does not need to be escaped
No - character % does not need to be escaped
No - character + does not need to be escaped
No - character e does not need to be escaped
No - character E does not need to be escaped
No - character f does not need to be escaped
No - character F does not need to be escaped
No - character g does not need to be escaped
No - character G does not need to be escaped
No - character h does not need to be escaped
No - character H does not need to be escaped
No - character i does not need to be escaped
No - character I does not need to be escaped
No - character j does not need to be escaped
No - character J does not need to be escaped
No - character k does not need to be escaped
No - character K does not need to be escaped
No - character l does not need to be escaped
No - character L does not need to be escaped
No - character m does not need to be escaped
No - character M does not need to be escaped
No - character n does not need to be escaped
No - character N does not need to be escaped
No - character o does not need to be escaped
No - character O does not need to be escaped
No - character p does not need to be escaped
No - character P does not need to be escaped
No - character q does not need to be escaped
No - character Q does not need to be escaped
No - character r does not need to be escaped
No - character R does not need to be escaped
No - character s does not need to be escaped
No - character S does not need to be escaped
No - character t does not need to be escaped
No - character T does not need to be escaped
No - character u does not need to be escaped
No - character U does not need to be escaped
No - character v does not need to be escaped
No - character V does not need to be escaped
No - character w does not need to be escaped
No - character W does not need to be escaped
No - character x does not need to be escaped
No - character X does not need to be escaped
No - character y does not need to be escaped
No - character Y does not need to be escaped
No - character z does not need to be escaped
No - character Z does not need to be escaped
Yes - character ¼ needs to be escaped
Yes - character ½ needs to be escaped
Yes - character ¾ needs to be escaped
Yes - character ¹ needs to be escaped
Yes - character ² needs to be escaped
Yes - character ³ needs to be escaped
Yes - character æ needs to be escaped
Yes - character Æ needs to be escaped
Yes - character ª needs to be escaped
Yes - character á needs to be escaped
Yes - character Á needs to be escaped
Yes - character à needs to be escaped
Yes - character À needs to be escaped
Yes - character â needs to be escaped
Yes - character  needs to be escaped
Yes - character å needs to be escaped
Yes - character Å needs to be escaped
Yes - character ä needs to be escaped
Yes - character Ä needs to be escaped
Yes - character ã needs to be escaped
Yes - character à needs to be escaped
Yes - character ç needs to be escaped
Yes - character Ç needs to be escaped
Yes - character ð needs to be escaped
Yes - character Ð needs to be escaped
Yes - character é needs to be escaped
Yes - character É needs to be escaped
Yes - character è needs to be escaped
Yes - character È needs to be escaped
Yes - character ê needs to be escaped
Yes - character Ê needs to be escaped
Yes - character ë needs to be escaped
Yes - character Ë needs to be escaped
Yes - character í needs to be escaped
Yes - character Í needs to be escaped
Yes - character ì needs to be escaped
Yes - character Ì needs to be escaped
Yes - character î needs to be escaped
Yes - character Î needs to be escaped
Yes - character ï needs to be escaped
Yes - character Ï needs to be escaped
Yes - character ´ needs to be escaped
Yes - character ` needs to be escaped
Yes - character ^ needs to be escaped
Yes - character ¨ needs to be escaped
Yes - character ÷ needs to be escaped
Yes - character × needs to be escaped
Yes - character < needs to be escaped
Yes - character > needs to be escaped
Yes - character ¬ needs to be escaped
Yes - character | needs to be escaped
Yes - character ¦ needs to be escaped
Yes - character ° needs to be escaped
Yes - character µ needs to be escaped
Yes - character ¯ needs to be escaped
Yes - character , needs to be escaped
Yes - character ; needs to be escaped
Yes - character ! needs to be escaped
Yes - character ¡ needs to be escaped
Yes - character ? needs to be escaped
Yes - character ¿ needs to be escaped
Yes - character · needs to be escaped
Yes - character ¸ needs to be escaped
Yes - character ' needs to be escaped
Yes - character ‘ needs to be escaped
Yes - character ’ needs to be escaped
Yes - character " needs to be escaped
Yes - character “ needs to be escaped
Yes - character ” needs to be escaped
Yes - character « needs to be escaped
Yes - character » needs to be escaped
Yes - character ( needs to be escaped
Yes - character ) needs to be escaped
Yes - character [ needs to be escaped
Yes - character ] needs to be escaped
Yes - character { needs to be escaped
Yes - character } needs to be escaped
Yes - character § needs to be escaped
Yes - character ¶ needs to be escaped
Yes - character © needs to be escaped
Yes - character ® needs to be escaped
Yes - character ™ needs to be escaped
Yes - character ¤ needs to be escaped
Yes - character ¢ needs to be escaped
Yes - character $ needs to be escaped
Yes - character £ needs to be escaped
Yes - character ¥ needs to be escaped
Yes - character € needs to be escaped
Yes - character * needs to be escaped
Yes - character \ needs to be escaped
Yes - character & needs to be escaped
Yes - character # needs to be escaped
Yes - character ± needs to be escaped
Yes - character ƒ needs to be escaped
Yes - character – needs to be escaped
Yes - character — needs to be escaped
Yes - character ‚ needs to be escaped
Yes - character „ needs to be escaped
Yes - character … needs to be escaped
Yes - character † needs to be escaped
Yes - character ‡ needs to be escaped
Yes - character • needs to be escaped
Yes - character ‰ needs to be escaped
Yes - character ñ needs to be escaped
Yes - character Ñ needs to be escaped
Yes - character œ needs to be escaped
Yes - character Πneeds to be escaped
Yes - character º needs to be escaped
Yes - character ó needs to be escaped
Yes - character Ó needs to be escaped
Yes - character ò needs to be escaped
Yes - character Ò needs to be escaped
Yes - character ô needs to be escaped
Yes - character Ô needs to be escaped
Yes - character ö needs to be escaped
Yes - character Ö needs to be escaped
Yes - character õ needs to be escaped
Yes - character Õ needs to be escaped
Yes - character ø needs to be escaped
Yes - character Ø needs to be escaped
Yes - character š needs to be escaped
Yes - character Š needs to be escaped
Yes - character ß needs to be escaped
Yes - character ú needs to be escaped
Yes - character Ú needs to be escaped
Yes - character ù needs to be escaped
Yes - character Ù needs to be escaped
Yes - character û needs to be escaped
Yes - character Û needs to be escaped
Yes - character ü needs to be escaped
Yes - character Ü needs to be escaped
Yes - character ý needs to be escaped
Yes - character Ý needs to be escaped
Yes - character ÿ needs to be escaped
Yes - character Ÿ needs to be escaped
Yes - character þ needs to be escaped
Yes - character Þ needs to be escaped
[me@mysystem]$

So how do you use the escape character?

Simply change the character either to the ASCII code or add a backslash before the character, e.g. you will change $ to \$.

Another way to list all the ASCII characters could be achieved using script below.

for i in {0..127} ;do
    printf -v var \\%o $i
    printf -v var $var
    printf -v res "%q" "$var"
    esc=Escape
    [ "$var" = "$res" ] && esc=NoEsc
    printf "%02X %s %-7s\n" $i $esc "$res"
done |

    column

The output will then look like below.

[me@mysystem]$ ./esc2.sh
00 Escape ''            20 Escape \             40 NoEsc @              60 Escape \`
01 Escape $'\001'       21 Escape \!            41 NoEsc A              61 NoEsc a
02 Escape $'\002'       22 Escape \"            42 NoEsc B              62 NoEsc b
03 Escape $'\003'       23 Escape \#            43 NoEsc C              63 NoEsc c
04 Escape $'\004'       24 Escape \$            44 NoEsc D              64 NoEsc d
05 Escape $'\005'       25 NoEsc %              45 NoEsc E              65 NoEsc e
06 Escape $'\006'       26 Escape \&            46 NoEsc F              66 NoEsc f
07 Escape $'\a'         27 Escape \'            47 NoEsc G              67 NoEsc g
08 Escape $'\b'         28 Escape \(            48 NoEsc H              68 NoEsc h
09 Escape $'\t'         29 Escape \)            49 NoEsc I              69 NoEsc i
0A Escape $'\n'         2A Escape \*            4A NoEsc J              6A NoEsc j
0B Escape $'\v'         2B NoEsc +              4B NoEsc K              6B NoEsc k
0C Escape $'\f'         2C Escape \,            4C NoEsc L              6C NoEsc l
0D Escape $'\r'         2D NoEsc -              4D NoEsc M              6D NoEsc m
0E Escape $'\016'       2E NoEsc .              4E NoEsc N              6E NoEsc n
0F Escape $'\017'       2F NoEsc /              4F NoEsc O              6F NoEsc o
10 Escape $'\020'       30 NoEsc 0              50 NoEsc P              70 NoEsc p
11 Escape $'\021'       31 NoEsc 1              51 NoEsc Q              71 NoEsc q
12 Escape $'\022'       32 NoEsc 2              52 NoEsc R              72 NoEsc r
13 Escape $'\023'       33 NoEsc 3              53 NoEsc S              73 NoEsc s
14 Escape $'\024'       34 NoEsc 4              54 NoEsc T              74 NoEsc t
15 Escape $'\025'       35 NoEsc 5              55 NoEsc U              75 NoEsc u
16 Escape $'\026'       36 NoEsc 6              56 NoEsc V              76 NoEsc v
17 Escape $'\027'       37 NoEsc 7              57 NoEsc W              77 NoEsc w
18 Escape $'\030'       38 NoEsc 8              58 NoEsc X              78 NoEsc x
19 Escape $'\031'       39 NoEsc 9              59 NoEsc Y              79 NoEsc y
1A Escape $'\032'       3A NoEsc :              5A NoEsc Z              7A NoEsc z
1B Escape $'\E'         3B Escape \;            5B Escape \[            7B Escape \{
1C Escape $'\034'       3C Escape \<            5C Escape \\            7C Escape \|
1D Escape $'\035'       3D NoEsc =              5D Escape \]            7D Escape \}
1E Escape $'\036'       3E Escape \>            5E Escape \^            7E NoEsc ~
1F Escape $'\037'       3F Escape \?            5F NoEsc _              7F Escape $'\177'

[me@mysystem]$




Popular posts from this blog

TrueCrypt on macOS X Mojave 10.14

If you have updated your macOS recently to Mojave otherwise known as verison 10.14 you may not be able to install the last version of Truecrypt in order to access your old volumes encrypted with Truecrypt software. This article will guide you to get this working on your MacOS v10.14 (Mjoave) . Download the package from  https://truecrypt.ch/downloads/  or  https://www.truecrypt71a.com/downloads/ . Find downloaded package using Finder in your HDD/Users/username/Downloads folder and will look like  TrueCrypt 7.1a Mac OS X.dmg . Open file location in Finder and open or double click on  TrueCrypt 7.1a Mac OS X.dmg . This will mount Truecrypt 7.1a and will have Truecrypt 7.1a.mpkg in it. Drag the package T rueCrypt 7.1a.mpkg and drop in your Downloads folder. From Locations in Finder you can eject your TrueCrypt mount. Now go to your Downloads location, find the file  TrueCrypt 7.1a.mpkg , right click and select Show Package Contents . Find the file Contents/distribution.di

Tools you need before you start using Terraform for orchestration infrastructure in cloud

Terraform one of the famous open source tool used for planning, deploying and maintaining infrastructure as a code and the beauty of this tool is that it works across various cloud service providers. Even though I personally like the AWS Cloudformation, it is AWS only and most probably it will always remain limited to Amazon Web Services. It has few advantages over other tools when using with AWS and will be the first to have new AWS features incorporated before other tools and APIs catch up. However if you are not limiting yourself to AWS only or have hybrid environment you may want to use a 3rd party tool compatible for all cloud infrastructures. Terraform being one of them is youngest client only tool and this article will help you setting up your desktop environment and give you understanding of how it works before you can start to code. Prerequisites: Computer running Windows, Linux, Solaris or MacOS operating system. Downloads: Download Terraform directly from the crea

Copy files and folders using SCP with spaces in path

Copying data from one system to other with file or folder names that contain spaces in path can be achieved using this guide. In this case I am copying data from Macbook to Windows 10 computer. In order to copy the data easily it is better to use bash commands. Windows computer can support WSL (Windows subsystem for Linux) and you can run one of few linux distributions to use shell commands. I have Ubuntu set up within my Windows 10 using WSL. If you do not have WSL, you can set it up using my guide here . The copy can be performed in two ways: 1) Using SCP Source (MacOs) path: /home/Users/username/Documents/data extract from 2020/ First of all you add escape sequence to the path so it will become:  /home/Users/me/Documents/data\ extract\ from\ 2020/ . While this works on local system for SCP you'll have to double the escape sequences by replacing \ with \\, as below. Figure out your source computer IP address using "ifconfig" command. Now using scp command on target syst