What if

For the last 2-3 years, Brian Aker and I have had many discussions about how to refactor MySQL. Brian has been the one driving these discussions by asking why some things in MySQL were done in a certain way and in a true "what if" manner asked what would happen if we would do things in another way.

Being tired of not being able to get critically needed reconstruction work done in the MySQL server, Brian started to work on Drizzle to search for answers to these questions.

So what is Drizzle?

• Drizzle is what MySQL would be with a more interactive community involvement in the design of the software itself, and had targeted website deployments.

• Drizzle is a version of MySQL that is driven by Brian and the community, attempting to solve practical problems that a large group of MySQL users are facing.
  

• Drizzle is a smaller, slimmer and (hopefully) faster version of MySQL; Features that the broad Drizzle community does not want or need are now removed or in the process of being removed (This includes stored procedures, views, triggers, grants, some non-pluggable storage engines and more).

• Drizzle is the 3rd fork of MySQL server code base, but is the one that has (for now) the most developers working on it. The two other forks are:
• ProvenScaling's fork on mirror.provenscaling.com provided by Jeremy Cole. You should read Peter Zaitsev's blog about this.

• The MySQL-5.1-maria fork, provided by the Maria team lead by me. (Brian did beat us when it comes to opening up the tree for outside development; We are still at least a month away from doing this).

• You can find out more about Drizzle at Brian's blog at http://krow.livejournal.com/.
  

Why is it that the people working on Drizzle are extremely enthusiastic about Drizzle?

This is because Drizzle solves many of the problems that MySQL's development has had for years:

• It opens up MySQL development for the community; You no longer have to wait years to get your patches and resonable extensions into the server.

• Critical bugs that have existed for years can finally get fixed as the development is no longer constrained by unrealistic release schedules that put artificial constraints on things that can be fixed.

• Drizzle will put some MySQL server differentiation on a true test; A bit like Fedora does to RedHat.

• Drizzle has created new excitement in the MySQL developer community; A lot of people seem to be very enthusiastic to work on it in a true community-oriented manner.

• Developers working on Drizzle is doing drastic refactoring of the server, something that MySQL planned to do years ago but never happened.

• Development decisions is again driven by people that are using the server daily; This will ensure that Drizzle will be faster and more stable than what can be done with current MySQL development model

• Drizzle will target the MySQL core users, the web users, whose requirements have been ignored for years while the core MySQL developers have added features that they don't need.

• In addition Drizzle will include the latest InnoDB code; You don't have to wait for MySQL 6.0 or go to the trouble of annually downloadoing and installing the InnoDB plugin from Oracle just to get access to the latest and fastest InnoDB version.

That said, Drizzle is not here to replace the normal MySQL server; Drizzle targets a limited but important market and will thus help us the enhance the MySQL based offerings. Think of Drizzle as the microkernel server around which other offerings/features can be developed.

Drizzle has no release schedule or timeline, but will follow a true open source and agile methodology of releasing early and releasing often; Brian expects that there will be a usable, reasonably stable version of drizzle within 3-4 months, but there are no promises.

Drizzle will also be of great help for the 'normal' MySQL developers; By looking at how Drizzle is solving things and by constant benchmarking against Drizzle, we will get a better insight into the weaknesses of the current codebase and have a better idea of what needs to be fixed; Friendly competition is good!

Drizzle is one of the good things that have been made possible by Sun acquiring MySQL. Brian has been working on Drizzle with the blessing and encouragement from Sun's upper management. We are finding Sun to be open and encouraging of innovation, this has been a good aspect of the acquisition.

My personal reaction to Drizzle is that I am very enthusiastic for this new kid on the MySQL block. I don't agree with everything that is done, but most things makes a lot of sense. I am looking forward to a lot of very interesting discussions about solutions in Drizzle that will help improve both MySQL versions.

A bugs life

This is a request to all MySQL users to help mysql developers, by providing information, so that we can help you, by providing a more stable MySQL server for your needs.

As you may know, MySQL 5.1 has been in state of release candidate (RC) for some time. The last RC was announced as the last RC and is supposed to be followed by a GA release. The GA release is planned to be the exact same code as the last GA, only with the label changed.

The question we, who are developing and supporting the MYSQL server have been asking ourselves is, "Are really now in shape to do a proper GA release?".

We would like you as a MySQL User to help us out with deciding this.

We don't want to repeat the mistake we did with MySQL 5.0 GA and then again with MySQL 5.1 RC, by releasing a MySQL 5.1 GA too early.

Our external criteria for General Availability (GA) or Production release can be found here.

What you may not know is that we have as part of our internal GA criteria, a requirement that we should not have any serious bugs, crashing or wrong result bugs, that affects a notable amount of users. The criteria states that it is ok to postpone fixes for bugs that have a low impact (ie affects few users).

This has the following implications:

• Bugs for which we don't have many user/customer reports for are not likely to get fixed. (In the worst case not even in 6.0 !)
• It's ok to go out with bugs in new feature in the GA as long as we don't have many users/customers that have reported problems with these features.
  

In other words, if you have an issue with a serious bug that exists in 5.1 that you *really* would like to have fixed soon now is your chance to influence our development!

Note that it's ok, and we want, you to also report bugs after we release the MySQL 5.1 as GA. The more users/customers commenting on a bug the more chance it will be fixed!

What I would like for people to do:

• Report every single bug that you encounter or know about in 5.1 into our bugs system.

• If the bug is already reported, please add a comment to the bug report that it affects you too.
• If you are a MySQL support customer, add either a note to the bug report that you are a customer and the bug affects you or send a request as a customer directly to the MySQL support personal (they are happy to take your reports!)
• If you have reported a bug a long time ago that has not been fixed and this is still important for you to get fixed, please reopen the bug/add a note in the bugs database that the bug is still relevant for you!
  

Note that you should also do this for bugs that you know of in MySQL 5.0 that are still open for MySQL 5.1 (or any bugs that are labelled to be fixed in a future release). Bugs that are in earlier version are also very likely to also be in 5.1 if the bugs database doesn't say otherwise.

Lets make the MySQL server bugs life's harder by giving us more information about which bugs are really important for us to fix. More information will help us make better decisions about how and when a bug should be fixed.

Bringing up the baby

At last we are code complete with Maria 1.5, the so called crash safe version of MyISAM. Note that code complete means that all features are coded, not that the code is 100 % bug free!

The code should be available soon in a bzr repository near you. We will also shortly (as soon as we pass all build tests) make a normal source and binary release of MySQL-5.1-Maria from here.

There is still a couple of minor bugs that we know about and we will fix them shortly, but nothing that should stop you from testing/using Maria. See the file KNOWN_BUGS.txt in the source release for details.

The current release plan for MySQL-5.1-Maria is as follows:

- The current release will be an alpha release. (Technically it could have been a beta release but because we have added so much new code we wanted to signal this by making another alpha release).

- If there is not any significant bugs found the would require a major rewrite of the code, we will for the next release fix the found bugs and make a beta release. This should hopefully be done August.

- We will continue doing one beta releases per month until there is no more redesign done and the new bug reports are down to a minimum. Then we will make a final RC and then a GA release (assuming everything goes well).

We have soo far been able to keep the opens bugs in Maria down to a minimum. This together with a quite large test suite make us, the Maria team, reasonable confident that we don't need many beta releases before we can come to RC and then GA. The more people that is downloading Maria to test and use it, the faster it will get stable. Please report any bugs you find and we will do our best to fix them!

At the same time we are merging MySQL-5.1-Maria into the standard MySQL-6.0 tree, where Maria will be a core part of the server.

The next step is to span of a MySQL-6.0-Maria tree in which we will start working on making Maria fully transactional and more concurrent.

Bug fixes that affects MySQL-5.1-Maria will of course be done in this tree and then be merged up to the MySQL-6.0 and MySQL-6.0-maria trees.

Those that are interested in known more about Maria, now and future, are welcome to my Maria talk and/or the Maria BOF at Oscon on July 24'th.

At Oscon Tim O'Reilly will interview Brian Aker and me on a keynote. . We will all do our best to make this a fun and unforgettable event!

Now it's time for me to go and have my well earned vacation. (I actually planned to go already last Monday, but I wanted to finish my part of the Maria project before going and now it's next Sunday...). At least I will get some rest from computers during the next two weeks as we don't have any electricity in my summer cottage up in Vasa.

Hope to see many of you at Oscon!

For a few dollars more

It's nice to be able to make a difference.

Since David Axmark and I started to work on MySQL we also took a strong stand against software patents. MySQL AB have been sponsoring several efforts to prevent software patents in Europa.

Now David and I are continuing to do this outside of MySQL AB. We just gave an economic sponsorship to the Patent Lens, who recently lost their main sponsor, so that it can continue it's important work in making the patents system more accessible.

Here is a short description of Patent Lens from it's founder, Richard A Jefferson:

We are working to generalize the Patent Lens as the informatics platform of the Initiative for Open Innovation, which aspires to render worldwide patent systems understandable and navigable, and to guide policy and practice reform. We intend to create a sector and jurisdiction agnostic resource that is open source, open access and free of cost.

The funding you may provide will keep a small core team going long enough to make it through these lean times and build new momentum.

If you are a philanthropist and if you care about open source software and don't have a love for the current patent system, I encourage you to join us in sponsoring Patent Lens. You can also try to get your company to sponsor.

David and I have also recently sponsored another company that works for Free and open source software, the Software Freedom Law Center.

We did it because we believe in what they are doing and we are greatful for all the help that Eben Moglen gave us in the past when we got into trouble.

Honor award at HP Linux forum 2008

I got yesterday a honor award at the HP Linux Forum 2008 in Helsinki, from FLUG (Finnish Linux user group) for "important work for the good of Linux" and for "working to keep the code open even when part of MySQL management has been of different opinion".

The major award went to Ubuntu Suomi for their good work in translating Ubuntu to Finnish and helping Finnish users setting up and using Ubuntu.

Linux.fi got the other honor award for their work on creating a Finnish Wiki for Linux users.

Arto Teräs, the spokesman of FLUG, said among other things, that "MySQL was one of the killer applications that brought Linux to the business world and to other places where Linux was not used before". You can find a video of his presentation here.

I am honored and happy for the recognition from FLUG. Thanks FLUG for the award and for HP hosting the event!

It was a well attended event (for being Finland); Based on the number of cars on the parking slot I would estimate the number of persons to at least 500. The one thing that I found missing was that Sun was not present at the event. (Note to myself; Need to fix this for next year)

Back to the future

I was yesterday attending the "Open Tuesday" Sun & MySQL event. One of the first questions I got from the audience during my questions & answer session was what is my take of the recent MySQL proposition of having closed source parts/modules in the server.

I was very happy to be able to say that Mårten some hours earlier had announced on CommuntyOne that the MySQL server is and is always going to be open source.

It looks like the community is also quite happy about the outcome, as can be seen on the responses from Slashdot when this was announced.

Mårten did also say on the CommunityOne panel that, that we should "expect Sun/MySQL to continue experimenting with the business model, and with what’s offered for the community and what’s offered commercial-only."

My hope is that the experiment when it comes to closed source extensions developed by Sun is now ended. As far as I know, there is no existing plans for any closed source extensions to the MySQL server. The strong commitments made now by Mårten and by Jonathan Schwartz in his interview by Tim O'Reilly where Jonathan said "Everything Sun delivers will be freely available, via a free and open license (either GPL, LGPL or Mozilla/CDDL), to the community. Everything. No exception." should ensure that things stays this way.

Kaj Arnö, who attended the event, has in his blog this in detail and his post is worth reading.

I would however like to clarify one thing Kaj said, in an attempt to avoid future confusion:

"Then for MySQL’s business model. To financially support MySQL’s free and open source platform, we have a business model which allows both community and commercial add-ons, and we remain committed to it."

I interpret this, in the context of Mårten's and Jonathan's announcements, that we will continue to support and make available commercial addons to the MySQL server from third party, like the Infobright storage engine. Things that we develop ourselves at Sun, at least on the server, will continue to be open source.

It's very good to see that Mårten is continuing to be responsive to the MySQL community and to the MySQL customers. Thanks to Mårten for doing the right thing! Thanks to the MySQL community for expressing their opinions!

For those that wonder what the title of the blog has to do with this post, I just want to say Open Source is the Future and the MySQL server is now back on track for the Future.

MySQL conference; The good, the bad and the ugly

The "2008 MySQL Conference and Expo" previously known as the "MySQL Users Conference" is over and I finally have time to blog.

What I really like with the MySQL conference is that it shows the loyalty that MySQL has from it's users and community. Year after year I get the change to meet the same friendly (!) faces and every year the group grows notable bigger (more than 2000 attendees this year). It really feels amazing for me to that I have had a small share in making this happen!

A lot of things happened during the conference, some good, some bad and some ugly.

Lets start with the bad parts and then go to the good stuff.

The Ugly

The ugly part was of course the announcement that MySQL was planning to change the MySQL server from open source/free software to crippleware by only giving out key parts of MySQL online backup (a server component) as closed source within the Enterprise server offering.

This announcement was not taken well by the MySQL community, as can be seen by the several hundreds of comments at slashdot

I would like to emphasis that this decision was made by the MySQL management team and *NOT* influenced in any way by Sun Management! All the Sun people I have met so far, including Jonathan Schwartz and my boss Greg Papadopoulos, CTO of Sun, has made it very clear for me that Sun is for open/free software and they was caught by surprise by this announcement. I think this happened because the integration process between MySQL and Sun is yet not complete, and MySQL AB is still functioning more or less as an atomic entity inside of Sun.

Mårten showed at his keynote a photo of where they were burning the IPO Prospectus for MySQL AB. This was a very cool thing to do!

What the MySQL management team forgot to burn, was all the plans they had of how to make more money when MySQL would be a public company. They have apparently not yet realized that when MySQL AB was acquired by Sun, things changed. As I understand, Sun is not interested in closing MySQL up, in fact quite the contrary. Sun bought MySQL to expand their open source/free software offerings and also to use the expertise we have in MySQL AB to transform Sun to be even more successful in this software space. More about this later.

The Bad

MySQL 5.1.24RC binaries (but not the source) was released without federated tables compiled in. Federated tables is a feature that is used by thousands of MySQL users, including MySQL customers. A lot of people within MySQL development, the MySQL community team, product management and myself, tried to stop this from happening but unfortunately we where not successful. I find it outrageous that something like this can be done within an RC release and see this as yet another indication that 5.1 is far from being ready to be released as GA. I think we are still 4-6 months from having a stable 5.1 GA.

The conference was a very good one, but one can clearly see the commercial interests are starting to take over the conference. There were fewer talks from the MySQL community than before, talks from people that could be seen as MySQL competitors was regulated and some talks was censored. If the conference is going even more into this direction the next year, the community should start thinking about organising a MySQL developer conference that would be targeting those that are contributing to MySQL server or developing solutions with MySQL.

The Good

The next release of MySQL 5.1RC binaries will again have federated enabled, hopefully with most of the federated issues addressed.

I agreed to join Sun one week ago. I will be working at Sun Labs directly under Greg Papadopoulos. My tasks, among other things will include:

• Finishing and Releasing the Maria storage engine as per current schedule
  
• Continue to do architectural work on the MySQL server and in general help with MySQL development
• Help Sun to be even more successful in the open source/free software world by actively helping different open source projects within Sun to be more involved with the community and more community driven.
  

I am very excited in joining Sun and working on making Sun the leading open source/free software company! I will of course make it a personal goal to keep MySQL development on track and to ensure that the MySQL server in the future will continue to be developed in an open source friendly fashion.

A new InnoDB version with a lot of new features was released under GPL at the conference for MySQL 5.1. Thanks to Heikki Tuuri and the InnoDB team!

My first talk "Maria engine" was quite well received. This included a short presentation of the XDB indexes that I have hinted about in one of my earlier blogs. A description of this will soon appear at http://forge.mysql.com/worklog/task.php?id=3537. We hope to finish the Maria 1.5 version, which includes concurrent inserts and selects to tables with versioning, within a few weeks. After that we will work on stabilising the MySQL-5.1-Maria release and then start working on the MySQL-6.0-Maria release that will include transactions.

You can find the slides to this talk at http://www.scribd.com/doc/2575706/Maria-Engine

The Maria BOF was even better received. Here we had a demo that proved that Maria was indeed crash safe and to celebrate the successful demo we then consumed 3 liter of black vodka.

My second talk "Future Design Hurdles to Tackle in the MySQL Server" or "The Future of MySQL (The Project)" was even better received. During the talk we had spontaneous chanting of "We don't ship crippleware" and afterwards I got many people coming to say to me that it was my best talk ever! (This doesn't of course say anything else than that I may be improving or that they have never heard me present before).

You can find the slides to this talk here.

I have been contacted to give this presentation again at other events, and I look forward to be working on MySQL with Sun, and with the MySQL Community.