WhiteRabbit's braindump

Killing undead processes

Wed, 14 Dec 2011 20:55:32 GMT

Sometimes you want to kill a process, and kill doesn't work, and kill -9 doesn't work either.

I think this can happen when a process is blocked in kernel mode waiting for some event.

What is sure is that a zombie cannot be killed. Zombie processes are already terminated, and appear in the output of ps as <defunct> (in Linux, anyway). The system keeps zombies around because their parent process is supposed to be able to receive their return value using wait(2) or waitpid(2), so the PID cannot be re-used until the parent calls one of these functions.

Just as in movies, you cannot kill a zombie but you can kill the zombie master: using pstree -p you can find the parent of the zombie, and that's the one you need to kill. As soon as the parent process dies, all zombie processes that it spawned will disappear.

You shouldn't feel too bad about killing the parent: leaving zombies is a sign that the application is malfunctionning anyway.

makemkv

Tue, 13 Dec 2011 19:56:00 GMT

Sometimes, you want to watch a DVD on a set-top box such as the Freebox (in a version that doesn't have a disc drive), or maybe you want to to watch an ISO...

I used to have mplayer scripts to try and extract the right track, with the right audio and the right subtitles. Until I ran into the Conan DVD ISO, which I just could not make sense of. Out of frustration, I set out to find a better solution. And here it is, Kevin!

Enter makemkv. It'll read the structure of the DVD (from the drive, or from an ISO), and let you create a MKV file, while selecting which audio and which subtitles you want to include in the final file. And it'll do all that in just a few minutes.

sslh 1.8

Fri, 15 Jul 2011 16:18:00 GMT

Just released a new version of sslh. It's beautiful and shiny, can connect to OpenVPN, is more RFC-compliant, and can run with select() i.e. without forking, which is supposed to be better for performance (especially under Cygwin).

Windows Backup

Thu, 09 Jun 2011 19:32:00 GMT

Ok, after a recent problem on Narelle's computer that suddenly exhibited signs of madness due to the Windows XP Recovery virus, I thought it was about time I installed a proper backup system for it.

My basic requirements for a backup system are:

Must work automatically (because users always forget to backup, or will "do it a little later).
Must work over the network (because there is no point doing the backup in the same computer, and saving to discs and both inconvenient (see point 1) and expensive).
Should be based on rsync or equivalent, because bandwidth isn't infinite.

In this case the additional requirements are:

Must work for a Windows client
Must work for a Linux server
Should only attempt backups when the Windows client is located on the local network, as we don't want to be performing a backup through a slow public wifi from the other side of the world. Although if optionaly we can do that (if we're travelling for months or something) it should be possible.

Because, well, that's just the way things are.

Last time I tried to solve this by installing Cygwin, a little script that would heuristically determine if it was on the local network (based on the IP, the gateway, and the presence of the server at the expected address) and rsync over ssh. This solution kinds of works, but Cygwin tends to be a last resort solution and suffers from Windows' lack of correct cron implementation, so scheduling is a pain.

There is actually a much simpler solution: Install the stand-alone version of rsync, configure it so rsync can access the Document and Settings folder and run it as a Windows service, then script everything from the server: a simple cron running every night, connects to the rsyncd service on the Windows client and retrieves the changed files. And then just feed that through the normal Unix backup you use (I use Backup2l).

You'll have to make sure the Windows computer inherits a fixed address from the DHCP server, if applicable, obviously.

This, combined with OpenVPN, could potentially be used to backup remotely (as in, over the Internet) as well; this is still to study, I'm just getting started with OpenVPN.

ClamAV failure

Sat, 05 Feb 2011 15:54:00 GMT

This morning Exim complained about ClamAV daemon not being present. Upon inspection it turns out that ClamAV was complaining that its databases were broken. This may be linked to all the restarts we made yesterday to test various combinations of programmes and see what made X crash. Anyway, removing the databases in /var/lib/clamav/ then running freshclam (which takes a while, as the entire database has to be re-loaded) fixed the problem.

It's unclear whether e-mail gets bounced or simply delivered without scanning when the malware ACL fails.

Of teaching languages

Sat, 05 Feb 2011 15:53:00 GMT

The French education minister got a new, brilliant idea to tackle the French' abysmal level of English (and foreign languages in general): let's start teaching children in kindergarten, from age 3. The rationale has been heard many times before: everyone knows that children are just like sponges, they learn extremely quickly and can just pick up foreign languages in a couple of days. In fact, it's known that we should also just teach them advanced mathematics and quantum theory. Logically, the next minister should try to out-do the current minister, and propose in utero English introduction courses (which will give the "introduction" a whole new meaning.)

Clearly this does not pass the mockability test. Many things bother me in this theory: children are supposed to learn a language easily, yet French kids are taught French in school for 10 years, and after that many are still not very good (as in, can't write a coherent cover letter that's not littered with grammar and spelling mistakes.) In the past I met a German guy who grew up in Brazil for a few years. He did in fact learn a lot of Portuguese. He left Brazil by the age of 6 or so, and when I met him in Germany as he was 35 he could not speak or understand a word of Portuguese. A lot of things that children are taught at a very early age are just forgotten if the teaching is not continuous.

And then there is the assumption that adults just can't learn new languages. I think that's a convenient position adults like to corner themselves in: they suck at languages, but what can they do about it, it's the fault of their teaching system that didn't start on them early enough, and now they're 30 and it's too late and they'll just suck at languages forever no matter how hard they try, so why bother. Well, that's just all bullshit: I know from personal experience that I started English in school at the age of 14, I absolutely sucked at school until I was about 20, discovered the Internet and started using English as a communication tool and became bilingual within a few years, to the point that native English speakers often can't tell I'm not in fact a native. I went on to become fluent in German as well, a language at which I sucked even more than English at school.

I think the biggest problem with the French language teaching system is a problem of goal (and this may apply to some other language teaching systems, but I can't vouch for them). English teachers tend to see themselves, I guess, as English literature teachers. As long as they don't start regarding themselves as communication teachers, their efforts are doomed. There is no point reading Marting Luther King when you can't order food at a restaurant. There is no point knowing how to describe a picture when you can't ask for directions to the nearest hospital. There is no point teaching English as a abstract subject without showing to kids that it can serve them to connect to the world, which is something that the Internet actually makes possible today (I do pity language teachers from the time of my youth, when the only easily available resources were Time Magazine and the Wall Street Journal.)

In a similar vein are the language teachers with an agenda. My wife teaches English in Engineering schools in France. She was once asked to teach an entire curriculum about the place of women in society and in the industry in particular. Way to go to bore 20 year-old engineering boys who would like nothing more than to have more girls in their classrooms, but have no idea why that's not the case. Meanwhile her fun class on creative use of technology was poo-pooed as pandering to the kids (I guess). Here's a teacher with the wrong agenda: her classroom is not the place to teach kids about feminism, it's supposed to be the place to teach them English, and the topics chosen should only matter insofar that they motivate the kids. Show them that understanding English enables them to understand Youtube videos about robots, and they'll actually go and seek them by themselves. Show them that English girls like Byron, and they may even get an interest in poetry.

On top of that problem of creating motivation (the carrot), there was a problem of lack of coercion (the stick). I remember being clearly told that in high school, it didn't really matter if I sucked at history, or biology, or languages, because I'd never have to do the year again as long as I was good enough at French and maths. Right, so I'm a 14 year old French boy, I see no point in learning any English because let's be honest, who the heck would want to live in Britain, and I won't get punished if I suck at English? Give me a break, I'd rather play video games. Uh-oh, that D&D Pool of Radiance is entirely in English. And that's how by the time I reached the end of high school, I could have taught English teachers about medieval weapons (in English, obviously). Still couldn't ask for direction to the nearest inn, though. That's the strength of motivation. This particular point might be slowly changing, with a raising number of Engineering schools making it mandatory to have a Cambridge certificate or a minimum TOIC score to obtain the degree. However, while that certainly should be motivating for Engineering students, there needs to be something similar earlier on that applies to all children (presumably, if you consider everyone should speak a little bit of English, which the minister's idea seems to entail.)

So there, another silly idea from a minister that probably doesn't know much about teaching, nor about languages. Luckily, there will be no funding for this idea and it'll stay where it is. Actually, want to bet that when they start finding it difficult and expensive to implement, they'll suddenly come across a study that explains that teaching kids languages at an early age achieves nothing?

ReadItLater, Pentadactyl and Kindle

Sat, 05 Feb 2011 15:53:00 GMT

You know how you always open too many tabs in Firefox for deferred reading, and then you end up with a thousand tabs because you're not getting around to reading them, and then Firefox crashes and then you lose them all? Plus, it's rather unmanageable.

Enter Read it later. Now you can just add the tab to a reading list, and close it. Later on when you have time, you can come back to your reading list.

But wait, there's more. You can open an account with Read It Later, and then you can synchronise your reading list between all your Firefoxes.

But wait, there's more. You can get Calibre to use your reading list as a news source (in the "unknown" section of Scheduled News) and Calibre will download your entire list, make it into a periodical entry, and send it to your Kindle. Now how's that for cool (and non-straightforward)?

Ok, so to get to that point I ran into a number of problems. Read It Later does not work with Vimperator, and I have no idea why. I discovered that there is now a branch of Vimperator called Pentadactyl (which is actually a funny name), which is compatible with Read It Later, and seems to have some new functions over Vimperator too. But Pentadactyl requires Firefox 3.6, which is not in Debian yet. I discovered Mozilla provides a tar of Firefox that just unpacks and somehow works directly on Debian stable. Problem solved: with Firefox 3.6, I can have Pentadactyl, which lets Read It Later works happily. And that's how wanting one little extension makes you upgrade half of your computer.

And on top of that, I add a file readitlater.js in ~/.pentadactyl/plugins containing:

commands.addUserCommand(['readitlater'], "Read Later",function(){RIL.addCurrent();RIL.addedFromLocationBar();});
commands.addUserCommand(['markasread'], "Mark as Read", function(){RIL.markCurrentAsRead();});
commands.addUserCommand(['readsomething'], "Read Something", function(){RIL.readSomething();});

That way, I have new commands: :readitlater and :markasread, to set my reading list, and :readsomething to open the reading list. The next step will be to integrate the reading list properly the same way as it is done in Pentadactyl.

Posting e-mail

Mon, 04 Oct 2010 16:51:00 GMT

So, recently we ran into a problem at englishintoulouse.com: we have over 500 members, and our ISP, which I used as a smarthost, objects to our sending more than 200 e-mails a day. So, sending a newsletter becomes a two-day job, with the last people on the list getting the newsletter a lot later than they should.

I went through the pain of setting up my own outgoing SMTP server. It was simple 10 years ago when all SMTP servers just accepted your e-mail: thanks to spammers, it's become much harder and your setup needs to be a lot more precise if you want large ISPs to accept mail from your.

Here are the changes I made.

SPF

SPF (Sender Policy Framework) is a DNS record that documents who has the right to send e-mail for your domain. In our case, the IP address documented as A and MX record are allowed to send e-mail for us.

For MaraDNS, you add a line like this:

%       spf     'v=spf1 a mx ~all'
%       txt     'v=spf1 a mx ~all'

More details about SPF, including a script that builds the SPF record for you, are available on the SPF Web site.

ISP setup

Several large ISP (e.g. AOL) filter incoming mail depending on the emitting IP address.

Configure a reverse DNS with your ISP, so the RDNS is consistent with your domain (I am not sure this is totally useful).

Remove your residential IP from SpamHaus PBL: , link "Remove IP Address". This lets most serious ISPs (Yahoo, Gmail, ...) receive your e-mail. Some other ISPs use a blacklist from Trend Micro. You'll also need to request being taken off their list.

AOL still won't listen, so use your ISP as smarthost just for them, adding to /etc/exim4/hubbed_hosts:

aol.com: smtp.free.fr

AOL also lets you set up a spam feedback loop, but I am not done with that yet.

Check your DNS setup

Another goal I had was to pass all of zonecheck(1) tests. This can be either run from their Web site or installed under Debian apt-get install zonecheck.

It pointed out that my DNS setup was klunky at best. So, what I need to do is tell my registrar that each of my domains has two nameservers, ns1.% and ns2.%, with the appropriate addresses. The registrar transmits that to the registry. The thing is, this will only work properly if your DNS is consistent with the registry, i.e. you have ns1.% and ns2.% A and NS records set up. In MaraDNS:

%       NS      ns1.%
%       NS      ns2.%

ns1.%           80.68.89.183
ns2.%           82.235.147.6

Then zonecheck insists that my servers should also serve TCP requests. In MaraDNS, in /etc/mararc:

tcp_convert_acl = "0.0.0.0/0"
tcp_convert_server = "80.68.89.183"

(with the appropriate IP address of course), and restart zoneserver (not maradns!).

Another pretty, Web-based, DNS tool that you can use is DNSCheck, which has the advantage of giving you an external view of your setup even if you do not have access to a machine outside your network.

Misc

You can test your e-mail system with

telnet relay-test.mail-abuse.org

This runs a series of security tests on the IP you're coming from and tells you the results. Ideal to find if you're accidently set up as an open relay.

OmniComment 1.2

Sun, 13 Jun 2010 19:56:00 GMT

OmniComment now supports RSS. I mean, ATOM. You can subscribe to the comments on this blog by going to any page of the site and subscribing there, as I haven't done what is requred on Tiddly wiki.

You can also download it.

OmniComment

Sun, 23 May 2010 11:55:00 GMT

I am releasing today the first version of OmniComment. People can now leave comments not only on here, but also anywhere on my site. It'll now be obvious that I have no friends ;)

It's available here: OmniComment

Old comments that had been posted on Haloscan will be re-installed soon.

In terms of installation, all the details for OmniComment are in the tar ball for the server side. On the client side and for TiddlyWiki integration, you'll need OmniPlugin, which works fine but doesn't yet have all the meta-data that TiddlyWiki suggests. It should be as simple as copying OmniPlugin in your TiddlyWiki, and adding a line in ViewTemplate to add the comment link. You will however need to change something in omni.cgi (search for "ugly hack", it'll be obvious what you need to do).

OmniPlugin

Sat, 22 May 2010 07:31:00 GMT

Interface Tiddly with OmniComment to get a comment page for each Tiddler.


var cgi_path = "/cgi/omni.cgi";
var server = "http://" + window.location.hostname;

config.macros.omni = {};

config.macros.omni.handler = function (place,macroName,params,wikifier,paramString,tiddler){ 
    
    var id = story.findContainingTiddler(place).id.substr(7);
    var hs_search = new RegExp('\\W', 'gi');
    id = id.replace(hs_search, "_");

    xh = new XMLHttpRequest();
    xh.open("GET", server + cgi_path + "?count=1&filename=wiki_" + id, false);
    xh.send(null);

    commentsLabel = "Comments (" + xh.responseText + ")";
    var commentsPrompt = "Comments on this tiddler";
    var commentsHandler = function(e) { 
        window.location=server + cgi_path + "?filename=wiki_"+id;
        return false;
    };
    var commentsButton = createTiddlyButton(place, commentsLabel, commentsPrompt, commentsHandler);

}


//

ViewTemplate

Sun, 16 May 2010 17:39:00 GMT

<div class='toolbar' macro='toolbar -closeTiddler closeOthers +editTiddler permalink references jump'></div>
<div class='title' macro='view title'></div>
<div class='subtitle'><span macro='view modifier link'></span>, <span macro='view modified date DD MMM YYYY'></span> (created <span macro='view created date DD MMM YYYY'></span>)
<div class='tagging' macro='tagging'></div>
<div class='tagged' macro='tags'></div>
<div class='viewer' macro='view text wikified'></div>
<span class='comments' macro='omni'></span>
</div>

<div class='tagClear'></div>

The problem with externalisation

Sun, 02 May 2010 14:03:00 GMT

I read much concern about how our privacy is threatened by all sorts of companies, such as Google (who knows everything about your secret desires) and Facebook (who knows all of your friends' birthdays).

My biggest concern relating to all these services is externalisation, which comes at the price of loss of control over the service or your data, or both. What happens if tomorrow Facebook decides to charge you 5 euro a month for their service? Chances are there would be an uproar at first, then most users would consider that it's too small a price to pay to worry about it. Yet they'd never have joined in the first place, had the service not been free in the first place.

Something similar happened to me recently: I used Haloscan to manage comments on this blog. Haloscan was free, easy to integrate and convenient: it did everything magically, filtered spam out, provided RSS feeds for comments and so on. And then inevitably one day they ran out of resources and slashed their free offers. I no longer have comments on my blog.

External services are like the dark side of the force: they are easier, faster, but ultimately not as powerful as doing things the right way, which is hosting everything for yourself. Only then do you know your service will surivive the stand of time. Well, as long as you pay for your ISP bill I guess.

Eben Moglen expressed somewhat similar thoughts (although mostly from the point of view of privacy, which is another facet to the control of your own data) in a very interesting, recent interview.

O'Sushi Bar

Mon, 28 Sep 2009 10:40:00 GMT

This refers to O'Sushi Bar in town, we haven't tried any of the others, as it seems this actually is a chain.

We had high hopes for O'Sushi when it opened: it was small, and was the first Japanese restaurant that didn't emphasise sushi (which is a small part of Japanese cooking) or skewers (which we're not sure even exists in Japanese cooking :) ). We could get ramen, tonkatsu and few other traditional dishes at very affordable prices.

A year down the line, the restaurant has become quite big, raised its prices on all dishes, and now does a lot more sushi. It's got a cute sushi train that's actually done by boats floating on running water in a gully. Unfortunately, that kind of sushi is only available in a "all you can eat" formula, and is very, very average.

Altogether we've been disappointed.

O Sushi Bar, 30 r Gambetta, 05 61 21 98 64

Multi-criteria sorting in Perl

Fri, 28 Aug 2009 18:36:00 GMT

This is so simple I want to cry.

You know when you need to sort a list following several critiria? "Sort the files by type, then sort those that have the same type by name, then by size".

Perl's sort and comparison operators (cmp and <=>) come in handy:

 @sorted = sort { $a->type cmp $b->type or $a->name cmp $b->name or $a->size <=> $b->size} @list;

Assuming of course the objects in @list have type, name and size methods available.

This reads litteraly as "compare a and b's types, and if they're the same then compare their names, and then compare their sizes".

Of course you should never compare file sizes, because size doesn't matter.

Investing in real estate

Mon, 20 Apr 2009 19:28:00 GMT

A few days ago in a group conversation, there was a discussion on a number of bad things about France, and in particular on how it was a scandal that landlords had such a hard time getting rid of tenants that don't pay. I kind of put a damper on the conversation when I said that financially weak people should not invest in real estate, prompting questions on whether I considered people who are capable of buying a flat are financially weak. Although I felt I knew exactly what I meant, my arguments weren't ready. They are now.

Investment

The foundation of investment is to keep some of your money and lend it to someone else in order to make a profit. The capital you invest may get lost or go down in value, so investors associate a risk with any investment, and ask for higher yields for higher risk investments.

For example, investing in government-guaranteed bonds has a low risk of your capital disappearing, and has a low yield (in the order of 2% in France at the moment). Speculative investing in the Asian stock market can yield phenomenal returns (15%, sometimes more) but with a high risk of losing some of your capital (50% from 2000 to 2001 for example).

One of the basic rules of investing in the stock market (i.e. riskier investments) is that you should only invest money you can afford to lose. These words were actually literaly said by an investment advisor I met a long time ago during the dot-com boom. It sounds like reasonable advice.

Investing in Real estate

There is no reason to treat investing in real-estate any differently. This means that before investing in a flat, you need to assess your risk:

You may not find a tenant (reducing your ROI to 0%)
The tenant may not pay its rent (reducing your ROI to 0%)
The tenant may leave and cause extensive damage to the
flat (sending your ROI in the negative)

It is true that French law is strongly in favour of the tenants. However this is known ahead of time by prospective landlords, and should not come as a surprise.

According to my investment advisor, you should only invest what you can afford to lose. This means that:

if you can't find a tenant, it shouldn't be a problem,
if your tenant doesn't pay rent, it shouldn't be a problem,
if your tenant damages your flat, it shouldn't be a problem.

Basically it means that you can afford to lose the mortgage repayment. Otherwise you're exposing yourself to unacceptable risk (it's like borrowing money to invest in the Asian stock market). Being exposed to unacceptable risk is what I meant by being financially weak.

Converting WMA to OGG

Sat, 14 Mar 2009 09:16:00 GMT

At the moment I have no CD drive in my main Linux computer. As a result I can't use my favourite CD ripping program (which is jack, for the record). I have a CD drive on my work laptop which unfortunately runs WinXP, so it can only produce WMA.

So, I use Windows Media Player to extract the tracks to lossless WMA, then I copy them over to the Linux server and convert them as such:

 for i in * ; do mplayer -ao pcm "$i"; mv audiodump.wav "$i.wav"; done
 oggenc *.wav
 rm *.wma *.wav
 rename s/.wma// *

and I'm left with the same files as OGG instead of WMA. Straightforward, almost.

Securely running Firefox in a chroot

Fri, 13 Mar 2009 19:23:00 GMT

First problem: Debian stable's Firefox is old (2.x). Solution: install a chroot containing Sid, and install the latest version.

Thing is, it's a recurring problem in Debian; I had removed my chroot settings last time Sid turned to Stable, and I forgot how it was done. This time I'll document it.

Second problem: The Web browser is one of the most critical pieces of software you use. The main assets on a computer are its functions (the programs and servers you run) and your data.

My machine acts as a server, so it's got Apache: but Apache can only read files, not normally write to them. So a vulnerability in Apache alone won't compromise my system much.

On the other hand, my Web browser has the same rights as me, which means it can read or write any of my personal files. A vulnerability in Firefox is a threat to all my data.

Solution: run Firefox under a different user ID which has access to nothing.

So, let's do it.

I use schroot as a wrapper to chroot.

# Setup chroot
debootstrap sid sid

# Use same users as main system
cd sid/etc
rm passwd; ln /etc/passwd .
rm shadow; ln /etc/shadow .
rm group; ln /etc/group .
ln /etc/hosts .

# Install Firefox
cd ..
chroot .
apt-get update
apt-get install iceweasel

Now outside the chroot we add the user (which automatically gets added to the chroot as well) and install schroot:

adduser me_www
apt-get install schroot
vi /etc/schoot/schroot.conf
<< configure >>

For some reason X forwarding won't work if the chroot doesn't share the same /tmp as the main system (this doesn't totally make sense to me) so we add this to {/etc/fstab}:

/home           /root/chroot/sid/home/ none bind 0 2
/dev            /root/chroot/sid/dev none bind 0 2
/tmp            /root/chroot/sid/tmp none bind 0 1
/sys            /root/chroot/sid/sys none bind 0 1

Don't forget to mount all that ({mount -a}) then we can start Firefox, in the chroot, under the new user ID:

schroot -u me_www -c sid -- iceweasel --display $DISPLAY

Simple, innit?

On Genetically Modified Organisms

Wed, 14 Jan 2009 14:41:00 GMT

A good article from a genetician commenting on the fears about GMO. Very moderate and reasonable. Except for the fact that it's in French.

Yakiyori

Sun, 16 Nov 2008 09:16:00 GMT

A small japanese restaurant. It's easy to get lost on the way, so make sure you have a map of Colomiers. The service is very slow, so don't plan to go to the movies afterwards. It's the best sushi we've tried in and around Toulouse. Budget about 35EUR a head.

Yakiyori: 14 r Prat, Colomiers, 05 61 15 79 21