Saturday December 30 2000

After a week offline, I came back to Ted Ts'o having found that my gzip-rsync patch didn't work for him. After some investigation, I discovered that I'd made a huge braino. here is the new patch, which gives the following results on my collection of consecutive .deb packages:
normal size = 31603726 (100%), rsync xfer = 29134042 (92%).
uncompressed: size = 69849664 (221%), rsync xfer = 20563916 (65%).
rsync size = 31750518 (100%), rsync xfer = 13132994 (41%).
As you can see, `gzip --rsync -9' is slightly larger than using normal gzip -9, but gives a much reduced transfer.

Tuesday December 19 2000

DaveM and Alexey still debating over the proper ip_defrag fix. PPC bug turns out to be a broken vmalloc (iptables seems really good for testing vmalloc, since we use it heavily). Closed off the FHS version 2.2, and am going to rework the layout while on holidays.

Ran through old netfilter bug reports and patches, doing scores for them. Wow, have I let that go in the last couple of months. Sorry. Also, made the scoreboard 2-column, and cleaned up the web page.

Monday December 18 2000

Moved house; that means no ADSL for a while as I get that moved across. Netfilter problems springing up; Linus shattered the makefile when he translated it (no surprise, it's had issues before), the fragment code reorg broke local collation of fragments (also no surprise, another prior problem spot) and finally there seems to be a ppc bug.

Alan is slowly merging my set_bit audit results with Linus, but hasn't reached the netfilter one yet (and another Sparc64 report just showed up). I'll send that separately.

Thursday December 14 2000

Finally sent off the first batch of overseas Christmas cards today. We actually made them by hand, so they're pretty crappy, but since there's about 45 minutes of work in each (I suck at craft), people had damn well better hold onto them. I'm so glad that I don't have some addresses: twenty of the damn things is enough.

We actually started making them back in November, but the enthusiasm waned. My fault.

Thursday December 7 2000

Lost my hard drive yesterday. Got a backup done just before, however, so very little lost. Still, I lost a day futzing around restoring: rsync keeps hanging doing the restore 8(.

I'm going to have to do some work on parallelism to fill my slots at linux.conf.au.

Wednesday December 6 2000

I announced the closing-off of FHS 2.2, for 15 December. It's not as quick as the LSB folk might like, but it allows for a 1 month review & discard process: expect FHS 2.2 just after 15 January.

Thursday November 23 2000

On plane (with a record 6 batteries) to Ireland. Talk is finished, but rsync still isn't (hopefully it'll happen before I return to Australia). Or maybe I'll just drink Guiness for my two days in Ireland.

Turns out I'm not alone in not knowing that `set_bit()' needs a long, and I sent a search-and-destroy patch to linux-kernel. Alan picked it up, so hopefully it'll be included in 2.4.

A bug was reported with 2.4's ipchains compatibility module; seems to be broken in 2.4.11, but I didn't get a chance to test it before I got on the plane (things have been really hectic).

Friday November 10 2000

Didn't get around to rsync implementation: that'll be next week I hope. Caught up on some netfilter work, got ADSL at home (much better than 56k modem!). Alison did the shopping online. Discovered that `set_bit(b, &x)' does not equal `x |= (1<<b)' on all architectures (ie. Sparc64): endianness issues. This breaks connection tracking on Sparc64, I've made a simple patch which, if it works, will go to Linus.

FHS work continues. Hmmm, maybe I'm optimistic for a rapid release of FHS 2.2...

The Linuxcare Ozlabs social club has some money, so we're looking at buying a video game machine (or two...). Chris Yeoh and I would really love a Gauntlet machine, but a cheaper option will probably prevail.

Monday November 6 2000

Debugged tdb, for real: no known bugs. Automatic hash table resizing is the only thing left, and it's something of a wishlist item (ie. we'll see how the implementation pans out). Did some more FHS work: I just want to close off the 2.2 version, not change the world, but I'm tempted to rework it majorly. Maybe next time.

Friday November 3 2000

More a case of `someone needed to do it' rather than aptitude: got involved in FHS 2.2.

Tridge came up with a much better method to make rsync scale: I'll be implementing that next week (basically, do one directory at a time). This means a version increment, so we'll fall back to the old code for older clients.

TDB work continues: Anton did the spinlock stuff, and I've done the endian-neutral work and lots of locking enhancements -- it's now impossible to deadlock the database (without using `tdb_chainlock()' which is only there to make Tridge happy). Watch for a 2.0 release in coming weeks (it's worth a major revision, according to local consensus).

OK, having written that, there's still a few bugs in the locking...

Tuesday October 31 2000

Accepted an invitation to speak in Dublin at the end of the month; should be fun, although going round the world in 9 days is not exactly a holiday...

Friday October 27 2000

Sent rsyncable gzip patch to the Debian gzip maintainer: he hasn't applied it yet AFAICT. Turned out to be relatively trivial (I originally assumed that the size of the window for calculating when to reset the Huffman tables would be much larger than the internal 32k window; this is false, and a value of ~16k is fine). Next step is to propose its use for .debs, then the world will be mine. Bwaahahaha!

Thursday October 12 2000

Still can't find netfilter OOPS: maybe it's the SMP VGA console problem in disguise? We'll see; now he's trying it with serial console.

Two new releases of apt-proxy this week: major new speedup, plus a one character fix.

Friday October 6 2000

Yesterday's patch didn't solve OOPS, it seems. I notice that ip_queue is insmod'ed, but the usage count was zero: I assume it was not actually used, otherwise I can simply blame JamesM 8).

I have preliminary gcc patches (against 2.95.2) for memset arg reversal warning and stricter enum checking. Here are the untested memset and -Wstrict-enums patches for the 20001002 CVS snapshot.

Thursday October 5 2000

Day spent doing work stuff, reading netfilter mail, updating web page. Got a report of an ASSERT getting hit in the conntrack code; obviously a connection was getting deleted from the lists twice. So I got out a pen and paper, wrote down all the instruction sequences which deleted connections, and walked through "what if this one runs at the same time as that one" until I hit not one, but two SMP races. Sent patch, but it'll probably take a week to know this is really nailed.

ipchains release had `make install' bug, reported by two people. Fixed and released under same release number (no source differences).

Speaking of which, we have to have a new iptables release. There's lots of stuff which could use a wider audience (patch-o-matic mainly) before people start using 2.4 in anger.

Wednesday October 4 2000

Reminder: write diary more often. Really.

Been busy:

Switching to 6:10 sleep for a while to try to keep up. Haven't read netfilter list in a week. Must do that tomorrow.

Monday September 18 2000

Argh, Linus slapped me on the wrist today. Too many patches. That means only the critical stuff gets in 2.4.0. The rest will have to be made available from the web site.

I'm going to make OUTPUT NAT a config option: it doesn't work at all at the moment, and it's fiddly. I broke it when fixing ICMP error handling a while back, and didn't detect it for months.

Friday September 8 2000

I'm out of the office today, flying to Adelaide for Iain and Melissa's wedding, so I hope apt-proxy on flame survives in my absence.

Hiroyuki Yamamori sent another mailful of corrections to the Packet Filtering HOWTO which he is translating into Japanese. He went so far as to look in the code to check the against text! So there may be more corrections to come.

Thursday September 7 2000

Spent Wednesday hacking on apt-proxy still: installed it on flame Wednesday night, which led to an afternoon of debugging today, interrupted by having to write the lecture I was giving for Operating Systems Implementation at ANU at 3:00pm. Lacked my usual panash, but my mind was still in shell scripts and rsync...

Hacking on --rsyncable option to gzip. The gzip maintainers are probably impossible to get changes past (gzip is ported to everything: the code is a mess), but I'm trying to make this as non-intrusive as possible. This is exciting because widespread deployment of this option when creating .deb files will reduce stress on the Debian mirror sites by an order of magnitude (once apt understands rsync backends: apt-proxy already does). In practice, this probably means that it will flatten the Debian bandwidth requirement curves for a couple of years as it gets rolled out.

Tuesday September 5 2000

Today I've been working on apt-proxy. New version will be tested on flame for a week or so, then I'll roll it out. I've nominated this as version 0.98. Expect an announce soon. Yes, it's still in sh.

Uncovered a bug on the testing with the Ultrasparc (thanks to Anton Blanchard), which can be fixed in userspace. Packet queuing doesn't seem to work at all though, leaving wide holes in the test suite.

Friday September 1 2000

Spent the last two days fixing up the test suite after the TCP changes, and getting it to run on the iMac (found two bugs: one minor one mine, and one major PPC one).

Two bug reports on ipchains within a week means it's time to roll 1.3.10; a very minor release. Sent the notification to the ipchains list, and the change contributors: the beta can be found here.

Big news is that Harald Welte made top of the scoreboard, by doing conntrack and NAT for IRC, miscellanous other patches, and ULOG, which I think will prove very popular among the more hard core users (it has the potential to become a significant project in its own right).

Tuesday August 29 2000

Looks like my ftp bug also bites the 2.2 (and presumably, the 2.0) ip_nat_ftp modules. Nasty. But because they don't drop packets containing a partial PORT pattern, they won't stop making progress until later (much less deterministic). Sent Juan Ciarlante a report.

It's interesting that two independent approaches hit the same bugs (I only ever glanced at the ftp code; most of this stuff I derived myself).

Saturday August 26 2000

Note to self: keep diary more up-to-date.

I finally tracked down that damn FTP-fails-after-30-files bug which some people were experiencing. I had to walk through the code by hand, line-by-line, substituting example numbers, and finally it hit me. This bug has been in there since the beginning, but only happened when your packets were being shortened, and the total subtracted bytes exceeded one packet size. Then, boom; ack numbers got screwed, and wierd things happened.

Felt good to nail that damn thing (easy once I managed to replicate it in the test suite).

Saturday July 29 2000

Half way through my trip; Amsterdam was great (met Pauline Middlelink and Rogier Wolff, as well as catching up with old friend Russell Coker and all-round-nice-guy Wichert Akkerman).

Since Coniston was booked out (sunny Saturday) as Alan Cox warned, Harald Welte (who gave me a lift from Manchester) and I are crashing here in the hall tonight. This place is much more of a chaotic user-group-meeting feel than OLS or any conference; arrived to see Duncan Grove, a friend of mine from my old home town, setting up ppp on the SparcStation 2 (the entire place is funnelling through a 56k dialup).

I'm running a public rproxy between my laptop and one of Harald's machines in Germany: it's really needed for this case, and serves as an excellent test. I also set up the DNS caching forwarder, then DHCP to enhance the peg-based system (and coincidentally, point them through the DNS proxy).

Sunday July 23 2000

OLS was great; missing Ali, but that aside it was fantastic. I really enjoyed all the talks I went to, especially Andi Kleen's and Jamal's talks. Mine was less thorough than I might have liked, since my idea of using a second laptop displaying the paper to remind me of the finer points failed.

Having to expand the conntrack/NAT architecture a little for helpers: avoiding allowing two connections to expect the same range of connections, for IRC and to fix a possible ftp problem, and also to add a `your expected connection has arrived' callback for videoconferencing protocols which must wait for another potential stream. Multiple expecteds off one connection will not be supported yet (although may well be needed later).

Friday July 21 2000

Finally met Wensong Zhang, of LVS fame. Like his new code for LVS on 2.4; it's completely modular and can go into the kernel at any time, although I want to play with the (fairly complex) reference count model to see if I can't simplify it.

Wednesday July 14 2000

Wow. A very long lapse in my diary. Here is my itinerary for the upcoming trip:
Sun 16 July 15:30AN 5968Canberra->Sydney (Arr 16:20)
Mon 17 July 9:35NZ14Sydney->Los Angeles
Mon 17 July 8:40NW932Los Angeles->Detroit
Mon 17 July 17:20NW3305Detroit->Ottawa (Arr 19:29)
OLS
Tue 25 July 15:00CP813Ottawa->Toronto
Tue 25 July 18:20KL692Toronto->Amsterdam (Arr 07:40 Wed)
Amsterdam
Sat 29 July 9:20UK2023Amsterdam->Manchester (Arr 09:40)
Linux Beer Hike
Mon 7 August 8:20UK2024 Manchester->Amsterdam
Mon 7 August 13:30KL1655 Amsterdam->Venice (Arr 15:20)
Linuxcare, Italia
Sun 13 August 06:45KL1650Venice->Amsterdam
Sun 13 August 11:50KL605Amsterdam->San Francisco (Arr 14:05)
LinuxWorld
Sat 19 August 19:35UA2067San Francisco->Los Angeles
Sat 19 August 22:15NZ15Los Angeles->Sydney
Mon 21 August 07:20AN5953Sydney->Canberra (Arr 08:10)

Tuesday June 20 2000

Woke up yesterday morning feeling human again. Spending the week at home to make sure I don't relapse; VCR is broken so I've been doing lots of hacking (body clock flipped to anti-phase, so there is very little else to do).

This puts me in phase with the Europeans on IRC, which is always nice (on the linux channels, there is less `chatter' when the US are asleep, and if I wanted to chat, I'd call someone up).

Did a trial implementation of RST generation, but it doesn't work with NAT, so I'm abandoning it again (the connection is unconfirmed, since the packet hasn't passed through the box yet, so the connection tracking code refuses to associate the RST reply with it). This could be worked around by generating the RST myself, but this either requires a very naive approach (easy to fingerprint), or dragging in a couple of hundred lines of TCP internals (no thanks!).

Thursday June 16 2000

It wasn't merely tonsillitis, it is glandular fever. Feels like someone ripped my throat out, and I'm told to expect that to last at least 2 weeks. Hacking from home, catching up on Email. Cancelled engagements at AUUG (even if I were recovered by then, I would still be too backlogged).

There's never a good time to get sick, but probability is good that I'll be fine in time for OLS.

Thursday June 9 2000

James Morris made the Core Team (congratulations James!). I just need to get organized so other team members can update the scoreboard, etc.

Monday June 5 2000

Worked both days last weekend, breaking my `no Saturday night' rule because I'm taking this weekend off entirely. Got alot of stuff done, various bugs fixed. Most importantly, I finally wrote a test for the ICMP handling and discovered some bugs (three, to be precise). It's always been problematic, and I should have had tests for it long ago.

But I'm finally up-to-date on EMail and most incoming patches. I want to get the scoreboard more uptodate, review bug reports, and run the test suite on the iMac (Paul Mackerras is back, so now it will surely behave). Pulled aside into some local testing work as well.

Friday June 2 2000

Another long day/night: it's almost 5am. While chasing the DoS problem, I've not been reading the netfilter list. Turned out to be a mistake: Russell King found two more errors (now patched). His old problems with ipchains I'm not sure about.

There's a bug lurking somewhere: had too many hard lockups on module removal to be coincidence.

Just found another bug: ICMP handling in compatibility modules (found as I was trying to update scoreboard through ipchains.o module: we run a 1460 MTU tunnel here, so those ICMP Frag Needed errors must be translated correctly!). Practice of Programming says `never leave a bug until later'. Make that 6:15am.

Thursday June 1 2000

Hacking late last night: managed to reproduce the `udpsic grinds box' problem. Profiling showed that it was not what I thought at all: turns out to be a massive hash collision in the NAT code. Came up with a complicated way around it, then finally figured that it was a waste of time: for the vast majority of cases (NAT to a single IP address), we don't even need to iterate the hash chain.

Even with my complex solution, you end up with the possibility of an attacker using fake IPs 64 apart (which hash the same), to get a huge hash collision. Harder, but still fairly easy. Anyway, if you're doing DNAT out to N servers, the attack will be spread over N chains, halving the debilitating effect (we store a hash of the REPLY src ip, dst ip and protocol in the NAT code).

Can't get a damn 2.4.0-test1 kernel to boot on the damn iMac. non-Intel platforms hate me, and our experts are all out of the office. Next week...

Monday May 29 2000

Kernel hacking and kernel locking guides into official kernel, Manfred Spraul sent excellent feedback. Netfilter pesky non-Intel problems remain (with all our non-x86 people out of the office, and me trying not to mess up their machines). Out of office for the weekend, so didn't get a chance to test my latest patches.

Thursday May 25 2000

Without Tridge to run CLUG, there was no organization for the meeting: I ended up doing an ad-hoc version of my OLS talk, only less technical, and it took about 90 minutes to get through half the material. I had no diagrams and only my 1 page of notes to go on, and no structude, but I think that most people enjoyed it (should have organized pizza after half an hour, which would have meant 1 hour talk).

Linus created 2.4 dir today: batton down the hatches.

Thursday May 18 2000

Cross-compiling is a bitch, and the SPARC Classic hangs regularly on pre9-2; NFS problems, random oopses, crashes. Had to patch the Sparc dir to support ethertap.

Tuesday May 16 2000

SPARC Classic apparently crashes on `iptables -L'. I have a SPARC Classic here, running 2.3.99-pre7. But Anton is in SF, so I don't know where he keeps his kernels. Want to put a new netfilter-enabled one on it (or maybe I'll leave that to him) and run the complete test suite. Hopefully it's just a userspace glitch.

Shorter day today: trying to keep body clock in sync with normality for Melbourne this weekend.

Monday May 15 2000

Got to sleep 7am. Good to be back at work.

Tuesday May 9 2000

More hacking, reading EMail, applying patches, keeping up with Linus, playing freeciv.

Have to get back to Canberra (my return is on Saturday).

Saturday May 6 2000

Updated the scoreboard finally, yesterday. I am so slack.

I've been putting off sending the `woah! break everything!' patch to Linus until I'm convinced that it fixes all those wierd platforms (fucking mac users!). I'm not in Canberra, so no access to Sparc or PPC.

Figured out rough itinerary today, for OLS and LinuxWorld:

Ottawa Linux Symposium: Wednesday July 19 - Saturday July 22
Linux Beer Hike: Sunday July 30 - Sunday August 6
LinuxWorld: Monday August 15 - Thursday August 17
Spent much time with friends (old and new) back here in Adelaide. It's been great, but I have to get back to Canberra to go into deep hack mode again.

Thursday April 27 2000

Sent off 6 patches to Linus today; I shouldn't queue for so long. Found the PPC problem: alignment. This kernel patch will break compatibility with userspace, but better now than later. I'll release 1.1 as soon as my patches get rolled in.

Sunday April 24 2000

Worked yesterday afternoon and this evening. Now I've got the test suite passing again, with all my kernel and libiptc changes. I have a TODO list again now (job tracking using the mailing list wasn't working so well).

Worked on an DocBook form of the Kernel Locking HOWTO. I'm not sure that it's suited for inclusion in the mainstream kernel, but I'll humbly submit it to Alan, and see what he thinks. Jeff Garzik wants me to submit the kernel hacking HOWTO the same way: I need to revise it heavily (remove the locking stuff, clear up the context stuff, etc) before I do that.

Paul Mackerras reported a bug in iptables on the PowerPC. Must get back to Canberra soon.

Thursday April 20 2000

Spent today speeding up libiptc, since I didn't like the caching approach done by Yon Uriarte: I prefer modifying the library design a little to make it more efficient. I now store all the chain names and lengths in a sorted array, and changed the way the iterators within a chain work (no random access to rule N of a chain). Should be much faster, and genuinely scale order N (not order N^2 like the current naive one).

Also looked at the code path used by tracking existing connections, and simplified it a little; cleaner code, fewer function calls. Profiling should help find furthur problems.

Wednesday April 19 2000

Time for some profiling; problem is putting enough stress on my boxes. But Ulrich Eckhardt reports that he can get it to hit 100% CPU pretty easily, so I'll try the same technique. It's not *that* vital, since I anticipate connection setup to be more expensive that connection association: since we only do it at most 1/10th of the time under normal conditions. Still, some if some judicious code-shuffling makes things faster, I'll do it.

Tuesday April 18 2000

Found the bug in my DoS protection code last night; now it's complete and merely needs testing. Good chat with Andi Kleen: we'd be better at avoiding Denial of Service attacks if we had TCP window tracking (we'd be able to tell which TCP connections were genuine, which we can't do at the moment).

After much trial and tribulation, got sound working on my Thinkpad 600E. Also, first physio today. Ouch. But no more self-administered injections!

Saturday April 16 2000

One thing that happened at IETF was that I met the organizer of the Singapore Linux Expo: Amlan Saha. That coincided with Richard Gooch's sharp-eyed note that Alexey had sent an Email from a .sg address: sure enough, Alexey was in Singapore! Unfortunately, this was confirmed too late for me to fly up there 8(.

I am now battling to get him to Ottawa in July: Amlan gave me a contact for the people who got him to Singapore, and I'm chasing it up with them now.

Getting Alexey in a room with Dave M, AK, and Donald Becker would do more for Linux networking than a month of me hacking out code. Hence I'm spending a reasonable amount of effort on this, even though I'm no travel agent.

Friday April 15 2000

Between the virus, the knee reconstruction, and the drugs (took me a few days to figure that it was the Codeine making me soooo tired), it's taken me a week longer than I'd hoped to get back to solid work again. I guess it's fortunate for me that the kernel doesn't look like hitting stability any time soon.

Some excitement last week with the Linuxcare announcement that the CEO was retiring: being out of the office I was probably the last person on the planet to know. I'm told we are still in the quiet period, so I can't say anything about it even if I had something intelligent to say...

The Wounded Knee Party (held in my absence at my apartment) is tonight: hope everyone has fun.

Monday April 3 2000

I feel awful: probably the virus which was running around IETF last week. And what a week. So much happened. So little time for mail and bugs. I go into hospital tomorrow morning so I'm furiously building a new kernel to run the testsuite against to send off last-minute patches to Linus, and release userspace tools 1.0.1.

Saturday I went out with some great local friends and Marc Boucher to a local club: didn't get much sleep, and injured my knee again (there's a reason I need tomorrow's reconstruction). Last night was milder: went out with Ted Ts'o and had dinner and coffee. Ted's always great value.

Tonight had Richard Guy Briggs and his wife Carol over my parents' place for dinner. Nice wind-down.

I'm on holidays this week. Yeah right: must hack.

Monday March 27 2000

IETF is cool; it's so wierd to have this stuff in my home town (Adelaide). Haven't read my mail yet (once I get this loan Wabvelan card working, Richard Guy Briggs has access to front to get the ipsec tunnel set up).

Sunday I finally found the `panic when builtin' stuff: sent it in to DaveM and netdev. Took out ipt_REJECT's tcp-reset option because (1) I didn't like it, and (2) I should have tested it before using the inbuilt kernel routines. Alexey has long felt that generating RST packets cavalierly is a real danger to the integrity of the protocol (as Microsoft did in this case).

Friday March 23 2000

Fixed all known problems with my netfilter stuff, except for Andi's reported ipchains-MASQ crash, and the `builtin-iptables-doesnt-work' mystery (the kernel Makefiles are fucked up). Fixed up the testsuite: James' ip_queue still broken, but he's rewriting it.

People keep asking me what happened to 6:10. Still going, after something of a lapse. I know I can do it now, so it's just a matter of deciding that I need to. I have needed to this week.

Wednesday March 22 2000

I used to think that I would one day figure out how to apologize well: my most heart-wrenching, deepest apologies always sound so thin and grossly inadequate.

It just occurred to me tonight that maybe apologies are *supposed* to be that way: so afterwards you don't feel any less sorry. Then you try as hard as you can to fix things.

Tuesday March 21 2000

Introduced everyone here to Spellcaster. Love that game.

Finished first cut of the NAT HOWTO, and working on the Networking Concepts HOWTO. Then the Packet Filtering HOWTO. Writing documentation is hard work, but this needs to be done soon. More patches from various people, and need to rework test suite as well, tomorrow.

Monday March 20 2000

OK. Merge is done. Userspace tools are done. HOWTOs being worked on. Minor kernel fixes flowing in to Alexey and DaveM (Dave took the weekend off).

At the conference, a John Adams turned up to talk with me, and his badge said `Netfilter'. Turns out that these guys have the trading name `netfilter' in Australia. Fortunately, it also turns out that John (one of their senior Sales/Marketing types) just wanted to talk to make sure I knew they were fine with sharing the name: he was pretty clued up, and obviously neither of us wants to get into a namespace war.

The end result is that we are swapping links on our pages, so if search engines send people to the wrong place, they don't get terribly confused. By the way, the do controlled (filtered, bandwidth-controlled, etc) ISP access for companies in Australia.

Friday March 17 2000

Hacking this week with all the hackers in the Canberra. Did the big merge with DaveM, but some last-minute fixes broke (I had a party tonight, and I needed to buy alchohol, munchies and clean up!), which he fortunately patched on the fly and fixed. Woohoo! Merged. Party time!

Monday March 13 2000

Caught up with Email. Out tonight for dinner (friend's birthday). Richard Gooch, Richard Guy Briggs, Ted Ts'o, Marc Boucher and Anton Blanchard are here now: Friday night we're having a party at my place, because we can. Wonder how many I can fit? We may be about to find out.

Saturday March 11 2000

Woah, what a week. Didn't read EMail all week, between doing my talk, time on the booth, catching up with everyone, the SLUG panel, much drinking, renewing an acquaintance.

I hear there's been some grumbling on the list about netfilter not compiling. Marc is here in Canberra now, and we're doing the final fixes before the merge.

Friday March 3 2000

Got in really late to the office: hung over after a celebratory night with Martin Pool on the eve of his trip to Apachecon. Did some damage to the liquor cabinet.

Tridge is back! I realized when I walked in and saw him at his desk today that there had been a big hole in the office without him.

Martin Pool gave me this quote out of ``The Reader'' which I just finished:

When an aeroplane's engines fail, it is not the end of the flight. Aeroplanes don't fall out of the sky like stones. They glide on, the enormous multi-engined passenger jets, for thirty, forty-five minutes, only to smash themselves up when they attempt a landing. The passengers don't notice a thing. Flying feels the same whether the engines are working or not. It's quieter, but only slightly: the wind drowns out the engines as it buffets the tail and wings. At some point, the earth or sea look dangerously close through the window. But perhaps the film is on, and the stewards and air hostesses have closed the blinds. Maybe the very quietness of the flight strikes the passengers as an improvement.

That summer was the glide path of our love.

Thursday March 2 2000

After discussion with a female ex-programmer friend on Wednesday night, I decided to remove a reference in the bibliography of the kernel-locking-HOWTO.

Between the discussion with her on the experiences she had, and a fascinating discussion with Telsa Gwynn on IRC, I'm sure that a woman coming to work at Linuxcare Ozlabs in a technical role would have issues that the guys don't. And I think the Linux Kernel has the same kind of issues.

Imagine if you wanted to be a coder, but gave up because others made it unbearable? That is the polar opposite of the spirit of Free Software, and I won't be party to it.

Wednesday March 1 2000

Finally, despite a big day of random distractions, started coding up the new conntrack stuff. The actual kernel patch proved to be much smaller than I thought (although the tunnelling code needs work).

Tuesday February 29 2000

Completed the first draft of my kernel locking HOWTO, which should provide minutes of amusement. I owe a big thankyou to Andi Kleen for making me read Schimmel.

Shot patch I off to the core team and James Morris for testing: if that's OK, I'll send

Monday February 28 2000

Did 0.90.3 release, then hacked late into the night on the laptop in the apartment, doing the kernel locking HOWTO.

Sunday February 27 2000

Still haven't gotten my laptop back from Sony, but I'm using an IBM Thinkpad 600E in the comfort of my apartment (which, unlike the office) has air conditioning on the weekends.

Can't believe that Igno and DaveM plan on introducing a new kind of lock this late in the 2.3 cycle. They must have an infinite supply of `get out of freeze' cards. Mind you, the brlocks are really cool, and I've changed the netfilter stuff over to use them, but fuck a brick, guys.

Saturday night, I went with David, Paulus and Martin Pool to a Bach Recital across the road from the office. It was really good, but I can't really tell the difference between great music and a great performance for that stuff. It was much fun though. Martin and I sought out the Canberra nightlife afterwards. I'm sure it's here somewhere.

Friday February 25 2000

Dave wants the entire netfilter all at once. OK, I don't feel too bad about that. It limits what massive changes we can do, but there aren't any left. Say 1 week for conntrack mods and netfilter changes, and then a couple of days to turn it into a kernel patch (requires neatening). Then prolly a week or so between Dave/Andi/Alexey and me to clean the corners.

Tired, but food (didn't eat lunch) and a shower should make me OK to go out with friends later tonight. Never turn down a chance to be sociable: you might not be asked next time.

Wednesday February 23 2000

Sometimes all you need to bring you out of a really bad day is the laughter and understandoing of someone you like. Had two nice phonecalls today: you know who you are, so (publically) thanks.

0.90.2 release compiling up as I write. ne2000 doesn't work under 2.4.47, so I couldn't commit while testing. Helped Paulus with a tty locking bug which has been plaguing PPP on 2.2 SMP kernels, which made me feel less useless.

Tuesday February 22 2000

Slept in this morning, after cutting in under 6:10 Sunday. Benchmarking is progressing: I cut down the masquerading timeouts so I didn't have to reboot between each test for 2.2.

Bookings for Chateau de Rusty are currently as follows:

3rd March
Martin Pool leaves for ApacheCon.
11th-22nd March
Ted T'so
11th-17th March
Richard Gooch (to be confirmed).
12th-24th March
Marc Boucher
Richard Guy Briggs
17th-21st March
Jes Sorensen
25th March
Martin Pool returns.
So it looks like we'll be renting some serviced apartments for the worst of the overloaded periods (12th - 22nd).

Sunday February 20 2000

Broke my rule, worked on Saturday. Had dinner with Richard, Simone (his SO) and a couple of others; lots of fun. Sunday afternoon caught up with James Morris, who's been doing some cool netfilter work. Got back, tired, had dinner, and kinda crashed. Still on 6:10.

Friday February 18 2000

FTP broken. Another wierd bug, but mainly seems OK. Performance testing soon, hopefully, but today I booked a flight to Sydney for dinner tomorrow night with Richard Gooch: after 2 years, one month and a few thousand flaming messages on linux-kernel, his devfs patch made it into the official kernel.

This is an event which must be marked.

Tuesday February 15 2000

Some months ago, Kylie wrote A Widow's Tale: it's an interesting view on hackers from an outsider's eyes. I'm not going to editorialize, as that wouldn't be fair.

Monday February 14 2000

Coding. Coding. Coding.

I want to have Larry McVoy's lovechild, after his linux-kernel thread. Larry is da man on scalability, and he used to work for SGI (and Sun, and...): here he is arguing with a current SGI engineer. My favourite is probably:

	... go talk to your hardware designers.  I know there are some left.

Saturday February 12 2000

After DaveM's hint to get my stuff merged, I'm breaking my own rule by working on a Saturday (there is nothing as depressing as a Saturday night at work). Running NAT testsuite now: reworked NAT expect stuff to be vaguely sane (although not completely).

Forgot to set alarm last night (set time, but didn't activate it), so I stole an extra 1:10 sleep. Still, I think that a Monday large-caveat 0.90 release is possible.

Friday February 11 2000

Fairly short day. Got up and into the office early, hacked. Sleep deprivation seems not to be effecting me in any significant way.

Martin's antennae are starting to bother me, but I don't think I should say anything about it: noone else has mentioned it so maybe he's sensitive. I can hear his evil avian laughter all the time now.

Thursday February 10 2000

Went for a little hike last night with Martin Pool and a local Canberran: a nice walk above the War Memorial (Mt Ainslie) to look down over Canberra.

Trouble getting up this morning, but nothing too severe: this 6:10 thing seems to be working. Netfilter testing continues: I keep adding new tests, so it's more gradual than it might otherwise be, but this isn't a race.

Testsuite runs up to the NAT tests again (on UP: SMP has oopsen which I suspect aren't my fault). A solid day: working all weekend might put me in range of a 0.90 alpha release.

Wednesday February 9 2000

Lack of sleep doesn't seem to be effecting me too much. Yesterday I was quieter than normal, but that may have been me being overly cautious. Got a fair amount of coding done though, in that extra time. Of course, watched Buffy and hung out on IRC, which is not why I wanted the extra time...

Was accepted for OLS : shaping up to be a fantastic conference. If I get ambitious I might go around the world in July...

Monday February 7 2000

Found ethertap problem. Another heavy day of working through (and adding to) the netfilter test suites. It's coming along pretty well: need to put in many more solid hours like today, though. It gets really hot in the office on warm nights.

I've decided to experiment with cutting down on sleep: I've decided to try a week on 6 hours, 10 minutes (I usually prefer an irregular 9). My coworkers have been warned that I might well be a raving psychotic by Wednesday.

Sunday February 6 2000

Went to Sydney again for some business late last week; drove back in Martin Nightingale's convertible late Friday morning. That was a fun experience, even though I got pretty sunburned. I guess it's something you have to do once, at least.

Still hacking on new netfilter generics. ethertap is broken in 2.3.42 still, so I can't run my test suite, but I can still break things pretty badly. At least my keyboard works under X again. DaveM shocked my by putting userspace checksumming in 2.3.41, and he tells me that softnet is going in as well. Linus can't be serious about 1 month to 2.4. Try 4 or 5.

Monday January 31 2000

OK, I've been lax. Last week I went to Sydney with some friends for Australia Day (last Wednesday) to go to the Big Day Out in Sydney. I had a really great time, and met some really cool people. Really good to get away for a bit.

Of course, coming back, I was right back into it with my CLUG talk `Rusty's Remarkably Unreliable Guide to Kernel Hacking', which was extremely chaotic. Slides up RSN.

Netfilter hacking continues: check something into CVS, and am now busy crashing my kernel. This will go on for some time before I can actually make a release.

Tuesday January 25 2000

Linuxcare is in the quiet period (hence the color change). This means that I won't be talking about any company-related things for a while in my diary. Netfilter coding, and the occasional personal aside are still in, though.

Packet filter generalization rewrite work continues. Slowly.

I am so much looking forward to hitting the road for a few days to get away; way more than I expected. Maybe this `holiday' concept has some merit. (OTOH, maybe after two days of not coding, I'll be itching to code again 8-).

Monday January 24 2000

netfilter patches flooding in from the mailing list: I'm ignoring them (except for an emergency 0.1.17 release) until I get the generic stuff committed, hopefully later this week.

Thursday January 20 2000

Very slow week, and next week's going to be worse (productivity-wise); I'm going to Sydney, and Martin Nightingale is coming to stay. Stayed late tonight putting humpty-dumpty together again for packet filtering. It's going to need an awful lot of testing, because there's MUCH new code there. Fortunately, not much of it is in the actual packet traversal, but the translation from userspace, etc (which are almost as critical).

The packet filtering kernel code now compiles again; I need to make NAT use it properly, and then the huge job of redoing userspace. I'm tempted to check in to CVS, but I have to decide how to rearrange files first.

Sunday January 16 2000

Had a long phone conversation with James Morris re: netlink and netfilter. Hopefully we'll see a patch come soon for getting rid of netfilter_dev.

Phone call was interrupted by Martin Pool dropping in to stay: I thought that was next week. Martin is joining Linuxcare, and rode his (motor-)bike down from Brisbane. Oh well, I hacked on Saturday (explicitly against My Rules), so I can't feel too guilty.

Friday January 14 2000

My TODO list seems to be growing longer. I may withdraw my request to Alexey, who doesn't like my skb-reservation patch. I can simply put a connection tracking pointer in the skbuff; that means a kernel merge, however. So the iptables generalization has to be done first; today I realized that it was not as trivial as I had started to think. Arbitrarily-sized iptables entries will please some people, however, since it means I can have a number of extended matches and (maybe) targets.

Submitted my proposal to OLS; I'm assured that it was really cool last year. Escaping winter in Canberra is always a good plan anyway.

Wednesday January 12 2000

Bad, bad Rusty. Got distracted in my generalization of iptables: started reworking the locking and counter addition code. Am undoing this, and will revisit it after I've put humpty-dumpty back together again.

Monday January 10 2000

Oh, we have a new Office Administrator, Tracy. This has been something of a shock for people calling up and now hearing a female voice at OzLabs. So far she's working out well: we may be able to delay the project to clone Hugh.

Sunday January 9 2000

So far, CVS has wasted far more of my time than it has saved. Dropping it entirely is looking more attractive by the minute. The documentation is pathetic for remote access to multiple repositories; I screwed up badly a number of times.

Today, releasing a new version (kernel change in 2.3.38 caused minor breakage) wasted over two hours of my time trying to figure out CVS (gave up, created manual diff).

I expect a learning curve, but this is crazy. I'm between a rock and a hard place here: I didn't want to master CVS (which you need to do, to work around its severe limitations), but I've given up on Larry McVoy's Bitkeeper project. I wouldn't be so severely pissed off if I hadn't had expectations that all this pain wouldn't be neccessary.

Friday January 7 2000

My TAGS patch finally got into 2.3.37!. Really, it wasn't that much of an ordeal.

Did an APM power-off hack which I'm not proud of; an SMP box here wants the APM BIOS calls done from CPU 0, and there's no simple way at shutdown to choose your CPU. I simply loop and create bogus kernel threads until I get scheduled on the right CPU. Linus will hate it.

Wednesday January 5 2000

Came into the office late this morning, to find everyone wearing black armbands. Turns out 2.3.36 was released, without my famous TAGS patch, even though Stephen submitted it to Linus as well. That's 8 and counting.

Monday January 3 2000

Nominally a public holiday, meaning I slept in this morning.

Found the skb bug: it was introduced by my patch in 2.3.35 (pointed out by Alexey). I found it on New Year's Eve; it was a simple matter of lazy coding (who would have guessed that skb_realloc_headroom was sometimes used to reduce headroom?).

Very little response from the plea for testsuite enhancements, but maybe that's because it doesn't run on RedHat (patch sent in by James Morris).

Today's coding task is to start on the generic-packet-selection CVS branch, off the main branch (it's orthogonal to the skb-reservation stuff). These two are the ingredients for netfilter-modules 1.0, which is a prerequisite for the merge to Linus.

Read through the core of the softnet stuff. I think I'd like to document what it is and what it changes now I went to the trouble of understanding it. Basically, it makes no difference to the netfilter module requirements (as Alexey said on netdev); even the netfilter internals remain unaffected, since I didn't try to be really clever.

Basically, Alexey implemented soft irq's: they run with no constraints whenever triggered (no serialization control). So far NET_TX_SOFTIRQ and NET_RX_SOFTIRQ use this. Under that he implemented `tasklets': these are guarenteed only to be run once on each CPU at any time. Finally, he also implemented bh's, which are never reentrant (ie. only run on one CPU at a time). I'm not quite sure why he didn't implement bh()'s as tasklets, but I'm guessing that it's for efficiency reasons: bh()'s are still going to be important in the forseeable future even if Alexey's code gets into 2.3.


Rusty's 1999 Diary