Blog

Blog

thunderbird / mailing list / reply

How do you reply to a mailing list post when you do not have the mail in your INBOX? With Thunderbird it is easy enough, as long as you know how. How it works Mail threads are matched by comparing the In-Reply-To header with the Message-ID. Here’s an example from an Asterisk project reviewboard mailing: Date: Fri, 08 Jun 2012 08:08:42 -0000 Message-ID: <20120608080842.8103.32910@hotblack.digium.com> In-Reply-To: <20120607143847.27705.11556@hotblack.digium.com> References: <20120607143847.27705.11556@hotblack.digium.com> Subject: Re: [asterisk-dev] [Code Review] Fix issue of unrecognized inbound ACK when Asterisk responds to an INVITE with a 481 Every e-mail message has a globally unique message identifier.

Read more

python / base85 / ascii85

So python’s base64 does not have a b85decode function? Adobe uses it the ASCII-85 encoding in PDF and PostScript files. Here is a quick and dirty one hacked together. See the wikipedia article for the ASCII-85 (base85) specs. Prologue; we only need sys to print a warning. # vim: set ts=8 sw=4 sts=4 et ai: # Example base85 decoder, Walter Doekes 2012 import sys Split the data up into 5-character chunks; 5 characters encode 4 octets.

Read more

libreoffice / spreadsheet l10n / date format

For LibreOffice’s oocalc on latest Ubuntu (libreoffice-base 1:3.5.2-2ubuntu1 to take the locale settings into account for the date types, the LC_CTYPE needs to be set. $ LC_CTYPE=en_US.UTF-8 oocalc This causes a date input of 31-01-2012 to not get parsed as a date. $ LC_CTYPE=nl_NL.UTF-8 oocalc This causes the same input of 31-01-2012 to get properly understood as the DD-MM-YYYY format. That does not make sense. LC_CTYPE should be used for character classification, collation and case conversion.

Read more

ubuntu / sip video / softphone

So, I wanted to test video support with Asterisk. That was easier said than done, because the SIP softphones that ship with Ubuntu don’t all do what they promise. This was done on a setup that works for numerous hardphones and PBXs out there. Looking through the registration list at any given time reveals at least 40+ different user agents and a large multiple of that if you take the different versions into account.

Read more

django / mark_safe / translatables

Look at this snippet of Django code in models.py, and in particular the help_text bit: from django.db import models from django.utils.translation import ugettext_lazy as _ from django.utils.safestring import mark_safe class MyModel(models.Model): my_field = models.CharField(max_length=123, help_text=mark_safe(_('Some <b>help</b> text.'))) For those unfamiliar with Django. A quick run-down: The definition of MyModel creates a mapping between the MyModel class and a underlying app_mymodel table in a database. That table will consist of two columns: id, an automatic integer as primary key (created by default), and my_field, a varchar/text field of at most 123 characters.

Read more

ipython classic mode / precise pangolin

The Ubuntu do-release-upgrade broke my ipython classic mode. The ipython package was upgraded, and apparently the configuration parser was changed. In bash, I want colors to help me find the beginning and end of output — see this bug report for others agreeing with me that the derogatory comment about “focus should be on the output, not on the prompt” in the skeleton .bashrc is is retared, but I diverge — in ipython, I just want to see the nice >>> blocks that I’m used to and no extra spaces.

Read more

safe_asterisk / init.d

An init.d script to stop and start safe_asterisk started asterisk. If asterisk is not stopped in 5 seconds, it is forcibly killed. safe_asterisk-init.d (view) # wget http://wjd.nu/files/2012/04/safe_asterisk-init.d -O/etc/init.d/asterisk ; chmod 755 /etc/init.d/asterisk Also possibly useful, the changes I made to safe_asterisk on a machine where: there wasn’t a tty left to spam output on, root is configured in /etc/aliases to a sane destination, /var/spool/asterisk is the asterisk user homedir anyway, and, attempting to set maxfiles to the highest value possible, wasn’t allowed.

Read more

sip / digest calculation

Every one in a while, I see an unexpected 403 response to a SIP client’s REGISTER request. Thusfar the digest response calculation has never been wrong, but it feels good to get that check out of the way and move on to other possible causes. For your enjoyment and mine, a Bourne-shell compatible shell script that calculates (qop-less) Digest authentication responses. Download: hahacalc.sh (view) $ hahacalc Usage: hahacalc.sh USERNAME REALM METHOD DIGESTURI NONCE [PASSWORD] [COMPARE] Or: hahacalc.

Read more

python virtualenv / global site-packages

If you’re switching from Ubuntu Oneiric to Ubuntu Precise and you’re using python-virtualenv, you might be in for a surprise: The default access to the global site-packages modules is reversed between virtualenv 1.6.x and 1.7. When you were used to finding your apt-get installed python modules like python-mysqldb and python-psycopg2 in your new virtualenv environment, now they’re suddenly unavailable. The culprit: --no-site-packages Ignored (the default). Don´t give access to the global site-packages modules to the virtual environment.

Read more

mysql / replicating repair table

From the MySQL 5.1 manual: 15.4.1.16. Replication and REPAIR TABLE When used on a corrupted or otherwise damaged table, it is possible for the REPAIR TABLE statement to delete rows that cannot be recovered. However, any such modifications of table data performed by this statement are not replicated, which can cause master and slave to lose synchronization. For this reason, in the event that a table on the master becomes damaged and you use REPAIR TABLE to repair it, you should first stop replication (if it is still running) before using REPAIR TABLE, then afterward compare the master’s and slave’s copies of the table and be prepared to correct any discrepancies manually, before restarting replication.

Read more

indirect scp / bypass remote firewall rules

Suppose I’m on machine DESKTOP and I want to copy files from server APPLE to server BANANA. DESKTOP has access to both, but firewalls and/or missing ssh keys prevent direct access between APPLE and BANANA. Regular scp(1) will now fail. It will attempt to do a direct copy and then give up. This is where this indirect scp wrapper (view) comes in: First, it tries to do the direct copy.

Read more

mysql replication / relay log pos

So, hardware trouble caused a VPS to go down. This VPS was running a MySQL server in a slave setup. Not surprisingly, the unclean shutdown broke succesful slaving. There are several possibly causes for slave setup breakage. This time it was the local relay log file (mysqld-relay-bin.xxxx) that was out of sync. SHOW SLAVE STATUS\G looked like this: ... Master_Log_File: mysql-bin.001814 <-- remote/master file (IO thread) Read_Master_Log_Pos: 33453535 <-- remote/master pos (IO thread) Relay_Log_File: mysqld-relay-bin.

Read more

mysql slow / queries / sample

Sometimes you’re in a situation where you know that a database is more heavily loaded than it should be. Time to figure out which queries are stressing it the most. The standard thing to do with a MySQL database would be to enable query logging with general_log_file. Or, to get only slow queries and those not using indexes, the log_slow_queries. But, if this is a mission critical and heavily loaded database, adding expensive logging may be just enough to give it that final push to become overloaded.

Read more

postgres / alter column / look closer

Just now, I tried to convert an integer column in a PostgreSQL database to one of type VARCHAR. I knew you had to do an explicit cast, so I was a bit stumped that I still wasn’t allowed to perform the ALTER TABLE. mydb=> ALTER TABLE mytable ALTER COLUMN mycolumn TYPE VARCHAR(31) USING mycolumn::text; ERROR: operator does not exist: character varying >= integer HINT: No operator matches the given name and argument type(s).

Read more

fixing symptoms / not problems

Some people seem to think that fixing the symptom is fixing the problem. import random def return_one_of(list): return list[random.randint(0, len(list))] def say_something(): try: print return_one_of(["Hello World!", "Hi!", "How you doin'?"]) except: return say_something() say_something() Gah! This is obviously an example, but there are people who do this and claim to have “fixed the problem”. Let me reiterate: the fact that your code does not raise any exceptions does NOT mean that it is not broken code!

Read more

django / mongodb / manage dbshell

The current django-mongodb-engine doesn’t seem to ship with a working manage dbshell command yet. Right now it returns this: $ ./manage.py dbshell ... File "/home/walter/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/management/commands/dbshell.py", line 21, in handle connection.client.runshell() File "/home/walter/.virtualenvs/myproject/lib/python2.6/site-packages/django_mongodb_engine/base.py", line 108, in __getattr__ raise AttributeError(attr) AttributeError: client The fix is simple, patch your django_mongodb_engine with this: --- django_mongodb_engine/base.py.orig 2011-11-15 11:53:47.000000000 +0100 +++ django_mongodb_engine/base.py 2011-11-15 11:54:07.000000000 +0100 @@ -7,6 +7,7 @@ from pymongo.connection import Connection from pymongo.collection import Collection +from .

Read more

certificate verify fail / crt / bundle

So. SSL certificates are still black magic to me. Especially when they cause trouble. Like when one of the sysadmins has forgotten to add the certificate bundle to the apache2 config. Then you get stuff like this: $ hg pull -u abort: error: _ssl.c:503: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed Most web browsers do not notice this as they already have the intermediate CA files, but /etc/ssl/certs/ca-certificates.crt seemingly doesn’t. The problem in this case was not that I was missing any certificates locally.

Read more

backtrace / without debugger

You may not always have gdb(1) at hand. Here are a couple of other options at your disposal. #1 Use addr2line to get the crash location $ cat badmem.c void function_c() { int *i = (int*)0xdeadbeef; *i = 123; } // <-- line 1 void function_b() { function_c(); } void function_a() { function_b(); } int main() { function_a(); return 0; } $ gcc -g badmem.c -o badmem $ ./badmem Segmentation fault No core dump?

Read more

gdb / backtrace / running process

Sometimes you want a backtrace or a core dump from a process that you do not want to stall. This could concern a multithreaded application of which some threads are still doing important work (like handling customer calls). Firing up gdb would halt the process for as long as you’re getting info, and raising a SIGABRT to get a core dump has the negative side-effect of killing the process. Neither is acceptable in a production environment.

Read more

sip / six digit port number / invalid

While looking through opensips logs of a customer, sometimes we would see the following: ERROR:core:parse_via: invalid port number <110900> ERROR:core:parse_via: <SIP/2.0/UDP 1.2.3.4:110900;branch=z9hG4bKabcdef... ERROR:core:parse_via: parsed so far:<SIP/2.0/UDP 1.2.3.4:110900;branch=z9hG4bKabcdef... ERROR:core:get_hdr_field: bad via As you can see, that 6-digit port number is invalid. Furthermore, when sniffing this traffic, we could see that the port number is almost right. The traffic came from port 11090 (one less zero at the end). Not only the Via header, the Contact header too had the extra appended zero.

Read more

openswan klips install / modules

If you want to be able to sniff your IPsec traffic with OpenSwan, you’ll need to get KLIPS instead of the default NETKEY IPsec protocol stack. Installing that on Ubuntu/Karmic should be a matter of: ~# apt-get install openswan-modules-source ~# cd /usr/src /usr/src# tar jxvf openswan-modules.tar.bz2 /usr/src# cd modules/openswan /usr/src/modules/openswan# make KERNELSRC=/lib/modules/`uname -r`/build module module_install But it’s not. Right now, we’re running the default Linux kernel 2.6.31-23-server on this Karmic machine.

Read more

mocp / random / enqueue

After disk failure on our company music server, I lost my enqueue-some-random-music-script. That shan’t happen again. So here, for my own enjoyment: autoenq.sh #!/bin/sh enqueue_app="mocp -a" music_glob="*.mp3" music_path="`dirname "$0"`" list_path="$music_path/.autoenq.list" if [ "$*" = "-c" ]; then # Create list of all files find . -type f -iname "$music_glob" > "$list_path.tmp" 2>/dev/null # no lost+found # Create list of all dirs that have files cat "$list_path.tmp" | sed -e 's/\/[^\/]*$//' | sort | uniq > "$list_path" exit 0 fi args="`echo "$*" | sed -e "s/['\\\\]//g"`" # no backslashes and single quotes please args="`echo "$args" | sed -e 's/[[:blank:]]\+/.

Read more

asterisk dialplan peculiarities / regex with eqtilde

In the Asterisk PBX dialplan, expressions can be formed using the $[...] syntax. Addition, subtraction, comparison and so on are defined. As is a regex operator: =~ Unfortunately, the documentation about the details of the implementation is hard to find. Here, a breakdown of my findings: static struct val * op_eqtilde is defined in main/ast_expr2.y It uses the REG_EXTENDED flag when calling regcomp: so extended regular expression syntax is used.

Read more

executing remote command / ssh / extra escaping

If you use ssh to run commands remotely, you may have run into the problem that you need an extra layer of escaping. Let’s say you have application myapp that for some reason only runs on host myserver. If you have functional ssh keys to log onto myserver it can be helpful to create a myapp wrapper on your desktop. After all, this: $ myapp myargs … is far more convenient than doing this:

Read more

django / query expression / negate

Suppose you have an is_enabled boolean in your Django model. class Rule(models.Model): is_enabled = models.BooleanField(blank=True) # other exciting fields here And now imagine you want to negate the is_enabled values. Something you would easily do in SQL, with: UPDATE myapp_rule SET is_enabled = NOT is_enabled; The Django F-syntax is nice, and looks like it should be up for the task. Let’s sum up a couple of attempts: Rule.objects.update(is_enabled=(not F('is_enabled'))) No! You get this:

Read more