The following are useful command line tips and tricks:
To count the number of files is a directory use the following command
ls -1 | wc -l
The option to the ls command is a 1 (one), the option the wc command is a lowercase L.
In bash use the seq command to add a range of numbers in a loop.
for i in `seq 10 20`; do telnet $i;done
Note back ticks around the seq command and arguments.
This particular example could be used as a simple port scanner.
To parse email log file extracting the To and From lines use:
grep -E '( from=| to=)' mail
Note the spaces before the words from and to.
To run a sed command over multiple files in a directory.
eg. htm files in a dir and output the changed files to a subdirectory called output/ use:
ls -1 *.htm | { while read i; do sed -f modify.sed $i > output/$i; done }This pipes the output of a single column ls command of *.htm files to a while loop that acts on each line of the ls output.
Each iteration of the loop runs the sed command using the modify.sed file that contains the modification commands.
The output of the sed command is redirected to the same filename as the input file but in a subdirectory called output/.
To change permissions on directories only within a file system use:
find /path-to-top-dir/ -depth -type d -name '*' -exec /bin/chmod -c 0604 {} \; Command options:
-depth
Do child directory before parent. This is important if restricting the w or x permissions as the restriction might stop the command moving into or changing a sub directory after permission is changed on the parent first.
-type
What to find, d for directories, f for files.
'*'
The ' ' is needed to stop error about the syntax.
{} \;ls needed to make the output from the find command an operand for the chmod command.
The ; is escaped by the \.
To run the command for files change the -type d to -type f.
Following command returns the size of the processor
eg. 32 or 64 bit.
getconf -a | grep LONG_BIT
This runs the get configuration command with the -a (all) switch. It then pipes the output through grep looking for the line that specifies the LONG_BIT.
For more information run:
man getconf
Using the -s option on the disk usage command du gives a summary of the disk usage by the directories below.
du * -h -s
Run this on the / directory might give something like:
--@--#du / -h -s
6.4M bin
6.0M boot
68K dev
4.9M etc
6.2G home
75M lib
du: `media/cdrom': No such file or directory
du: `media/floppy': No medium found
0 media
94G mnt
29M opt
65M proc
76K root
12M sbin
1.7M srv
0 sys
0 tmp
1.6G usr
136M var
Using an additional -x option means du will not read from mounted file systems. Handy to avoid the /mnt directory.
Over time a crontab file can become a complex and vital part of a GNU/Linux system.
If, while editing a crontab file, any issues occur it could be lost and may not be easily recoverable.
Using the following procedure can help to avoid disaster:
Use the -l switch to list the current file and redirect this to a text file
crontab -l > current-crontab
Edit this text file. eg:
vim current-crontab
This text file is now a backup of the crontab for later reference.
The file will contain lines at the top containing a warning etc. These lines can be removed, or left to indicate an update history as they contain a date and time that the crontab was last updated:
# DO NOT EDIT THIS FILE - edit the master and reinstall.
# (current-crontab installed on Sun Feb 25 09:53:24 2007)
# (Cron version V5.0 -- $Id: crontab.c,v 1.12 2004/01/23 18:56:42 vixie Exp $)
MAILTO=xxx@xxx.xxx
# Perform backup at 2am every day
00 02 * * * /usr/local/bin/backup.sh
# Backup via Rsync the firefox bookmarks files
46 01 * * * /usr/local/bin/rsync-bookmarks.sh
Once the current-crontab text file has been edited we need to install this file as the new crontab. This is done as:
crontab current-crontab
To confirm the changes have been applied run the list option (-l) again:
crontab -l
On a GNU/Linux system there are 3 standard sources for input and output called:
STDIN (Standard IN) = keyboard by default
STDOUT (Standard OUT) = screen by default
STDERR (Standard ERROR) = screen by default
These can be referred to by the numbers:
0 = STDIN
1 = STDOUT
2 = STDERR
To redirect the output of a program to /dev/null use the > redirector
wobble > /dev/null
This will redirect the information from the wobble program away from the screen to the null device; /dev/null.
However this does not redirect the errors, they would still be output to the screen. To also redirect errors we must:
wobble > /dev/null 2>&1
What this does is redirect STDOUT to /dev/null and it also redirects STDERR (2) to wherever STDOUT (1) is going, currently /dev/null.
During the transfer and setup of the PyeNet Universal website to the new SilverStripe CMS a useful use of SSH and the diff/patch commands has made the updating of files on the production server easier.
Once modifications to files on the development system have been made, those modifications are then transfered to the production server using the following procedure.
Performing the updates via the patch mechanism allows for the easy reversal of patches if something does not go well.
The steps are:
1. One thing to note is this process is designed for modifications to a small amount of files at a time. A proper Version Control system would be needed if a lot of changes are to be made to a lot of files at one time.
2. On the development system, change to a common directory located above all the files just modified. Running the following command will create the patch file:
ssh <user>@<production_system> "cat /full/path/to/file" | diff -u - relative/path/to/local/file >> output_diff_file
Command explained:
3. The created patch file now needs to be copied to the production system. It needs to be located in the same directory on the production system as the directory the command was run on the local system.
4. On the production system run:
patch --dry-run -p0 < output_diff_file
This command:
5. If the above command does not report any errors then run it without the --dry-run option to actually make the changes.
Note that the files patched will have their ownership changed to the person running the command. Confirm that these do not need to be changed back to their original owner.
Another use for this command could be for the maintenance of system configuration files on multiple hosts from a single host.
When a filename, or the path to a filename, is being used in the scp command the usual shell escaping mechanism of using a \ does not work.
To allow paths and/or filenames that have spaces to be used quote the path AND use the \ escaping.
For example
This will NOT work:
scp username@server:/directory\ name/with\ a\ filename\ with\ spaces .
The following WILL work
scp username@server:"/directory\ name/with\ a\ filename\ with\ spaces" .
HTH.