Cricalix.Net

Well, one of the things I expected to happen with the job change last year was the return of my energy to write code for random projects.  After several years of coding for a living, any desire to pick up a keyboard and hack on either my own projects ( or public projects ) was pretty much non-existent.

I’ve spent quite a few hours over the past week or so cleaning up the Value Editor mod for the EVE Killboard, along with patches to the core code, such as implementing the ability to store raw mails and fixing the portrait grabbing code.  I’ve also spent several hours providing support on the killboard forum for various other users of the software, and I have to say, I’m quite content now.  There’s something about providing code (for free) that people find useful.

Not sure what I’ll tackle next (other than the value editor, it needs a few more tweaks) – perhaps the mod that shows ship fits as a graphic.  One of the forum users has reported that it whacks their Apache installation load wise.

Yep, weird with a capital W.

Take two tables.  Un-optimized indexes are present on both tables.  EXPLAIN says 45 rows will be retrieved from one table, 5 from the other.  The 45 will invoke where, the 5 invokes temporary and filesort.

Alter the index on  the table returning 5 rows (it’s a pivot table).

Now explain says 75, 4, with 75 invoking where, temporary and filesort and 4 invoking where.

Alter the index on the table returning 75 rows so that the index hit picks up the data that’s getting pivoted.

Now explain says 158, 4 with 158 invoking where, index, temporary, and filesort and 4 invoking where.

The weird part?  The last modification is the fastest, even with no_sql_cache invoked.  More rows to be worked with (arguably it’s still a small number), but it’s faster.

A few years ago, when defining a table, I wasn’t very savvy to the fact that MySQL uses 4 byte pointers for the MyISAM data files (by default). This has subsequently bitten me in the rear a few times, enough for me to learn about altering the table with a max_rows argument, and to define the default pointer size in my.cnf. Unfortunately, the $work installation of MySQL has a bug – the default pointer size in my.cnf gets ignored. This leaves me creating all of the various tables with a manually specified max_rows to ensure that I don’t get bitten.

Enter a legacy table from a few years ago that just hit the pointer limit. The only solution I had in my arsenal is to rebuild the entire table with a larger pointer size. The last time I did this, it took 3 days. 3 days to rebuild the indexes for a 4 GB table. This time around, I tried a method that involves creating a new table definition without the indexes but with larger pointers, copying in all the old data, copying the old index file to the new index file, and executing a repair table. Unfortunately, while this method is meant to be faster, it only works when you can fit all of the keys into memory, otherwise it uses a slower rebuild method. 2 GB of RAM isn’t enough to hold all of the keys :(

So, chatting with Dad, he started to say something that tickled the grey cells. Namely that MySQL has a merge table format that can merge two (or more) identical MyISAM files, and support reads from both underlying files, while specifying where the writes should go. This means I can create a new underlying table with the new pointer size, then overlay the merge and specify the new table as the target for the writes. Total time to perform – 10 seconds, most of which is typing! Much, much better than 3 days.

Designing web interfaces is something that gives me grief. I’m not exactly the most elegant web designer in the world – interfaces that look like a bunch of stacked rectangles are fine as far as I’m concerned. They may not be pretty, but they are functional. Unfortunately, when doing $dayjob stuff, it has to be presentable to our customers, and they’re not always as technical as I am, so the interface needs to be ‘pretty’.

For the longest time, I’ve usually sketched a rough outline on a bit of paper, and then tried to implement the basics of it in HTML and CSS. I then hand it over to our actual web design guy to make sensible and prettier. Enter, stage left, QT designer. Yep, it’s a GUI UI designer, but it has drop-downs, text labels, text inputs, check boxes and radio buttons – which is exactly what a web browser has access to when rendering your bog standard HTML.  One print-screen later, and I have a nice mock-up of what I want the interface to look like, rather than a badly drawn version.  It’s also a heck of a lot faster to drag and drop UI elements than sit and churn out HTML code.

So I’m sitting here working on a mid-sized array that defines permission flag names, and the default values for different types of user – I’ve dumped the array out after sorting it, and I’m re-importing it into the code so that I can find bits easier (alpha search by analogue eye).  As I’m running a series of search and replace functions, I found myself thinking ‘Self, this would be easier if you stored it in an SQL table.’  I actually proceeded down that train of thought for about a second, until I remembered – this array in code is the source for a database table that the rest of the code uses.  It’s just the installer if you will.

*twitch*

$self had the clever idea that we could use the sort of new ‘Web 2.0′ buzzword paradigm of Ajax on the software $dayjob produces. All well and good, the boss approved, and the mock-up worked – a simple form, Prototype as the Ajax library and some PHP (5.2 no less) that returned a JSON string in the header. Adding records happened in the background, and on success, the page updated, a little status field updated, a reload of the entire data set from the database server wasn’t needed and life was groovy. All of the content validation occurred on the server side, and because it’s in a library, I only have to write one set of rules, regardless of whether it’s Ajax, standard forms, or even SOAP.

Part the next was to integrated the entire thing into the real code that drives our web interface. ‘No problem $self, you know it works…’ Well, who tell me think dat!? After some fiddling with the templating engine (which uses { and } heavily), and some re-writing and re-forming of the proof-of-concept code into a generic routine that could handle any of the configuration pages, I had a test page ready to rock and roll. The code set the Ajax URL target on the fly in the template, and Firebug reported that ‘Yes Bob, the data went out, and the response came back.’, but nary an update occurred on the page.

Over an hour later, after stripping the code down to barebones, making copious use of the debugger statement that Firebug can pick up, and at least 2 cups of tea, I realised two things.

1. I had missed three semi-colons on code inside of the var myajax = new Ajax.Request( ); segment.
2. I had used the wrong name for two functions, also called inside that block.

If you do that outside of the object creation, the JavaScript interpreter in the browser yells at you loudly. If you do it inside the object creation, it fails silently. Not an error anywhere. Not in Firebug. Not in Firefox’s error console. Nada. Not a sausage.

I now remember why I dislike JavaScript. Hopefully the Ajax book I ordered will give me some insight into how to deal with stupidity (on my part, and on the part of JavaScript) like that.

Some days, programming can be quite fun. Boss dropped a hot rock on the lap of myself and the other developer, saying that a customer (to be?) who had a bit of paper for us wouldn’t give us the paper until a certain feature offered by sales was done. Our task (and we didn’t get cool theme music or ’should you choose to accept it’) was to create this feature by the end of the week. By the end of the day would be even better.

Round 1 – see if any of our current stuff will do this feature without heavy bending. Nope.

Round 2 – come up with an idea, bounce it off of my fellow developer. Why not prototype it?

Round 3 – write prototype program in about 30 minutes. Test it lightly, declare proof-of-concept functional.

Round 4 – integrate round 3 into the existing code and test it some more. 30 minutes later, declare code functional as version 0.01.

Round 5 – package up the new code and store it somewhere safe should we have to do this again.

Total time from concept to functional, actually semi-robust code? 1 hour.

I like perl. I also like Net::Server.