A few years ago, when defining a table, I wasn’t very savvy to the fact that MySQL uses 4 byte pointers for the MyISAM data files (by default). This has subsequently bitten me in the rear a few times, enough for me to learn about altering the table with a max_rows argument, and to define the default pointer size in my.cnf. Unfortunately, the $work installation of MySQL has a bug - the default pointer size in my.cnf gets ignored. This leaves me creating all of the various tables with a manually specified max_rows to ensure that I don’t get bitten.
Enter a legacy table from a few years ago that just hit the pointer limit. The only solution I had in my arsenal is to rebuild the entire table with a larger pointer size. The last time I did this, it took 3 days. 3 days to rebuild the indexes for a 4 GB table. This time around, I tried a method that involves creating a new table definition without the indexes but with larger pointers, copying in all the old data, copying the old index file to the new index file, and executing a repair table. Unfortunately, while this method is meant to be faster, it only works when you can fit all of the keys into memory, otherwise it uses a slower rebuild method. 2 GB of RAM isn’t enough to hold all of the keys :(
So, chatting with Dad, he started to say something that tickled the grey cells. Namely that MySQL has a merge table format that can merge two (or more) identical MyISAM files, and support reads from both underlying files, while specifying where the writes should go. This means I can create a new underlying table with the new pointer size, then overlay the merge and specify the new table as the target for the writes. Total time to perform - 10 seconds, most of which is typing! Much, much better than 3 days.
