aggregated thinking

I was looking for a quick and painless way to determine what IP addresses used our web app the most and stumbled across this excellent post. I got reminded of the power of the command-line interface and that often, I underestimate the way these commands are supposed to make a sysad's life easier.

So here's a quick tip for counting unique visitors to your web app. Locate your web app's log file and do a quick scan of the last few lines. In my case, this is what I saw:

112.198.79.223 - - [10/Jul/2010:00:57:29 -0400] "POST /v2/api/execute.php?method=upload_time&ver=win1.1.15 HTTP/1.1" 200 116 "-" "Mozilla/5.0"
117.241.112.254 - - [10/Jul/2010:00:57:30 -0400] "POST /v2/api/execute.php?method=timestat&ver=win1.1.15 HTTP/1.1" 200 288 "-" "Mozilla/5.0"
80.249.84.105 - - [10/Jul/2010:00:57:29 -0400] "POST /v2/api/execute.php?method=offtime&ver=win1.1.15 HTTP/1.1" 200 19 "-" "Mozilla/5.0"
117.241.112.254 - - [10/Jul/2010:00:57:32 -0400] "POST /v2/api/execute.php?method=get_defaults&ver=win1.1.15 HTTP/1.1" 200 77 "-" "Mozilla/5.0"

In this log format, the IP address appears first, so we will only need to get the first set of characters, separated by a space. In order to count instances of an IP address, we sort the IP addresses so similar IP addresses are grouped together, then count them. Then we sort them again so that the sums are arranged. The number of unique IPs may give you pages of standard output, so optionally, if you only want to see which IP addresses accessed the web app the most, we look at the top results only.

So these are the commands used, piped one after the other, with the output:

[root@server2 ~]# cat /var/log/httpd/access_ssl.log | awk '{print $1}' | sort | uniq -c | sort -nr | head

71612 216.157.78.237
34094 120.28.205.244
30091 119.93.97.54
25078 120.28.214.104
18949 120.28.195.151
17929 58.107.64.40
14877 212.98.174.235
14752 117.241.112.143
14429 120.28.247.74
12864 117.241.113.183
[root@server2 ~]#

Happy counting, err reporting!

Some of my students reported that they were having problems connecting to their repositories using TortoiseSVN. At first, quick checks from my end didn't reveal what caused these, but I realized soon that it wasn't a server issue. It seems like the client software that they were using (TortoiseSVN) didn't know what to make of the svn+ssh://[user]@[ip]:[port]/[repo] URL.

I use RabbitVCS on my Linux boxen and get the same "Network connection closed unexpectedly" message. A colleague at work suggested that since a non-standard port for SSH is used, I should modify my /home/<username>/.subversion/config , look for the [tunnels] section, and append -p [port number] for the svn tunnel named ssh. Since this particular repository is just one among the dozens I access, I can't modify the default svn tunnel. Instead, I added a custom tunnel for each of the servers that use non-standard SSH ports.

For the repo that my students were trying to access, I used ssh2212 = $SVN_SSH ssh -q -o ControlMaster=no -p 2212. So the URL now looks like svn+ssh2212://[user]@[ip]:2212/[repo] and I was able to connect successfully.

In Windows, there are several ways to do this. Since the OS doesn't have a native SSH client executable, users will rely on Cygwin, Putty, or on TortoisePlink.exe (in newer versions of TortoiseSVN).

You can try any of the following solutions (they're all searchable in the Internets) and see if you can get around the problem. I've tried them all on a Windows guest OS on VirtualBox and only the last three solutions worked.

• Using Putty: Use saved session name from Putty in the URL: svn+ssh://[user]@[saved putty session name]/[repo]. Make sure that you configure the session to already use the correct port.
• Using Putty: Alternately, use IP:port format for the Putty saved session name so that you still use the svn+ssh://[user]@[ip]:[port]/[repo] URL format, but you're actually calling the saved session name.
• c. Using client packaged with Tortoise. In TortoiseSVN settings -> Network, change SSH client to "C:\Program Files\TortoiseSVN\bin\TortoisePlink.exe" and use standard svn+ssh://[user]@[ip]:[port]/[repo] URL.
• d. Same as d., but for the SSH client, append -P [port number]
• e. Using a wrapper script, such as the one provided here (excellent solution btw), and use it as the SSH client executable in TortoiseSVN settings -> Network.

Hope any one of these work for you. As an inside joke from one of my sysad friends, it was also suggested that I save myself the troubles and just log in to the server and revert to the default SSH port. LOL.

This post about copying TweetDeck settings in Windows explains this in detail, and I have been able to do the same for Ubuntu, replicating TweetDeck settings across 3 machines. Just take note of some of the differences that I'm going to point out here.

Adobe Air for Linux uses the .appdata directory in your home directory, for profile settings, configurations, etc. In the computer that has your groups and is the source for the export, change working directory into .appdata, and look for the TweetDeckFast.{some hash here}.

jon@jedi-ntbk:~$ cd .appdata
jon@jedi-ntbk:~/.appdata$ ls
Adobe
cookie_file.txt
de.makesoft.twhirl.0EA062BC275E7ED1E6EC3762EFFD73C7158ADF33.1
jrcf_H7t6Ap
jrcf_RA5I7w
jrcf_wdLW48
Snippage.B28FB424FD6880E47B18D7D649F6CC93BDE9B29B.1
TweetDeckFast.F9107117265DB7542C1A806C8DB837742CE14C21.1
jon@jedi-ntbk:~/.appdata$

Change directory to the one that says TweeDeckFast and list the contents. Change directory into Local Store and you should find two files that has your username. In my case, they are

preferences_jjdoblados.xml
td_26_jjdoblados.db

Simply copy these two files to any target machine in the same directory where you got them. Make sure that TweetDeck is closed before copying. When done copying, start TweetDeck and you will have all of your groups intact!

PIIX3 cannot attach drive to the Secondary Master.
VBox status code: -102 (VERR_FILE_NOT_FOUND).

This thread in the forums explains that the virtual machine was looking for a disk that was supposed to be mounted and could not find it.  I was using an external DVD drive and had removed it before I shut down the VM.

You'd get the same problem if you set up a VM to use a mounted disk or some other mass storage device.  To fix this, simply uncheck the non-existent devices in the VM settings while the VM is powered down.  Alternately, you can plug the devices back in before you power up the VM.

If after doing these steps you still get the error, check if the devices do get mounted and are accessible from the host as you may be dealing with broken or non-functional devices.

up for this week is my dabbling with stuff around kicker, and using kblogger to do some much needed updates.

1. Started a clan network. Okay, so most of them are really old and have no appreciation whatsoever for the internet, much less understand what emails are, and personalized email addresses at that. So I'll leave this up to the younger ones, who will help me teach the older ones the basics. They are yet to be informed of this new responsibility. LOL.

2. Upgraded desktop computer to Feisty Fawn Beta, and dual-booted Feisty and XP on my Pavilion dv2001tu lappy. So far, I have not had any hair-tearing frustrations with Feisty. Devices seem to work off-the-box, and the quick keys also seem to work fine (except of course for HP QuickPlay, which doesn't do anything). Touchpad, wireless, pcm+pcm2 with muting, and built-in mic all work. I've managed to get the built-in webcam to run with Ekiga (more details in a future post).

3. Started on my own list of what works and what doesn't for the HP Pavilion dv2001tu. Will try to make this as authoritative and as non-biased as possible. I love them Acer, Lenovo, and of course, Compaq notebooks too, but since I exclusively use my dv2000, I won't be able to vouch that the hacks I apply are going to work with the others.

4. Worked on more and more domains and hostings. I eat hostings for affiliate sites and stuff, day in and day out. I've optimized apache and mysql for several half-a-million traffic/day sites. Log analyzers are your best friends. I recommend Splunk for those who want to dig through files like search engines. But there's always the basic awk, grep, and myriad of other commands you can script together for the job.

5. Promised myself I'd start updating more often. 'Nuff said. =)