Match and replace TAB

I am using the terminal inside emacs to modify and clean textfiles (colon separated) for import into MySQL – mostly with SED, GREP, AWK.
Replacing TAB in a textfile with SED was not as straightforward as I thought. So Google came up with an excellent post on macworld.com and here especially one post suggesting perl as a solution. It turned out perl behaves inside the pipe the same way as SED/AWK.

Replacing TAB ‘\t‘ with colon ‘,‘ in a textfile ‘SomeFile.txt‘ can be done with:

perl -pi -e 's/\t/,/g' SomeFile.txt

or using the pipe, which can be combined with grep, sed, awk, tr, head, tail, etc…

cat SomeFile.txt | perl -pi -e 's/\t/,/g'

*Very* similar to the sed invocation, except for the flags. “-pi” for “*print* every line out after making the changes, while editing *inplace*. That is, no backup file is kept. In reality, a temp file is written, and if all is well (ie the operation succeeds), the temp file is shifted “onto” the original file, giving the appearance of an in-place edit.

Advertisements

MySQL command line with OS X 10.6.

After installing MySQL 5.1.54 on Mac OS X 10.6.6 (Snow Leopard) I tried to access the server in the terminal and got:

-bash: mysql: command not found

Bash could not find the path, so I had to add it. I am not that familiar with the internals of the unix terminal anyway, so I found a highly informative thread on superuser.com.

Especially the post of Jim Logan solved the issue:

Take a look at the file /etc/paths, which is used by /usr/libexec/path_helper, which is used by /etc/profile. For MacPorts, use “sudo” to append “/opt/local/bin” to the end of the /etc/paths file and reopen the terminal window.

So I opened the terminal and used nano:
sudo nano /etc/paths

and appended the line
/usr/local/mysql/bin

to the file, saved, exited, restarted the terminal …. and voila, solved.

Using gammu to connect to a mobile phone

After switching between mobile phones several times I always lost some data, contacts, media, so on…

There is a Linux tool called Gammu which allows to connect a selection of mobile phones. It seems that Gammus functionality is maximal for Nokia and Siemens, but I will give it a try on my Sony Ericsson…

The configuration is not trivial and I found some hints on JohnMcClumpha.org:

Install Gammu

Installing gammu is surprisingly easy (once again thanks to the wonders of apt-get), just use the following command:

sudo apt-get install gammu

Hard wasn’t it? 😉

OK now it’s time to plug your phone in and see if we can get things talking. With the phone connected, type the following command:

lsusb

you should now see your phone listed as a device – for example:

Bus 001 Device 002: ID 0421:0802 Nokia Mobile Phones

if not – make sure your cables and power are all good and try again.

The gammu installation comes with some example configuration files which are worth using as a starting point – if nothing else they help you to understand how gammu can be configured so that you can tailor a solution for your needs. These are located in
/usr/share/doc/gammu/examples
(in gZip archives).

Copy the gammurc file to /etc/gammurc :

sudo cp /usr/share/doc/gammu/examples/config/gammurc /etc/gammurc

Now edit /etc/gammurc to specify your port and connection type (this will vary based upon where/how you have things plugged in and what sort of cable/interface your phone is using). The settings for mine are:

port = /dev/ttyACM0
connection = dku5

Save this config and from the shell type:

gammu --identify

you should now be presented with some information regaqrding your phone such as:

Manufacturer : Nokia
Model : 7200 (RH-23)
Firmware : 3.110 T (18-03-04)
Hardware : 0903
IMEI : 353363000813894
Original IMEI : 353363/00/081389/4
Manufactured : 04/2004
Product code : 0514143
UEM : 16

If this is the case then you have got gammu up and running and can send yourself a test message with the following command:

 echo "boo" | gammu --sendsms TEXT [recipient mobile number]

Restructuring the filetree – moving files from multiple directories

I am considering changing the structure of my /home directory, maybe completely changing my data organisation habbits.

I found this post on ubuntuforums very useful:

You need to use find

For this example I`m going to assume that all the .txt files are located in directories and subdirectories of your Documents folder and you want to move them to a directory in your home called scripts
Code:
find ~/Documents -name '*.txt' -exec mv '{}' ~/scripts \;

Passing an external variable to AWK

Confronted with a heap of colon separated text files which had to be merged and cleaned of unrelated lines and columns, i tryed my luck inside Excel and spend a lot of time doing it manually, but finally got fed up.

So I decided to use AWK on the task.

A FOR-loop lists the files in the folder into the UNIX pipe.

AWK selects the non-empty observations and adds the name of the file as a classifier to the beginning of the line (the result is a repeated measure dataset).

This is the code:

for CSV in `ls`
do
cat $CSV | awk -F ";" '{
if ($2 ~ /[0-9]+/) {print CSV , FS , $0;}}'
done

Remark: -F ";" option specifies how to distinguish the columns/fields of the lines/records in the file(default is ” ” or empty space).

BUT: The variable CSV gets not passed to AWK by default it has to be fed into AWK.

Solution:
The
-v CATEGORY=$CSV
option feeds the external variable CSV into the AWK-variable CATEGORY.

This gives:
for CSV in `ls`
do
cat $CSV | awk -F ";" -v CATEGORY=$CSV '{
if ($2 ~ /[0-9]+/) {print CATEGORY , FS , $0;}}'
done

.. and works 🙂

Hat tip:fpmurphy

Mount a CD Image on Harddisk

A copy of a CD containing e.g. installation software or a game (so called iso image) can be mounted directly from the harddrive. This might be useful to mimic a CD without inserting the CD into the (very slow and power consuming) CD/DVD-drive.

1. Create a mountpoint e.g. in the /media directory:
sudo mkdir /media/iso

2. Mount the image MyCDimage.iso as a CD:
sudo mount -t iso9660 -o loop MyCDimage.iso /media/iso

3. Display all filesystems to check its mounted:
df -h

In order to play e.g. Windows games with wine the mountpoint /media/iso has to be added as a drive under wine.
winecfg
choose the drive tab and add /media/iso