Within Monty Program Ab we have during this year had a lot of discussions about how to go forward with the MariaDB trademark. It's been clear that everyone wants to have something that is substantially freer than the MySQL trademark to ensure the survival of MariaDB whatever happens to Monty Program Ab.
We wanted to make something that should work well, both for open source and commercial usage (and yes, I know that in some cases these are one and the same), which is not very common with many other trademark policies. My belief is that by having a very liberal trademark policy we will create a bigger ecosystem around MariaDB which will help all of us.
Now we have had a couple of internal drafts (with heavy input from our community advocates) and we have released our first public draft.
Please give us feedback about this either on my blog or the knowledge base so that we can take your thoughts into account for our final version!
2010-12-21
2010-12-16
A quick look at MySQL 5.5 GA
Congratulations to Oracle and the MySQL team for getting 5.5 out the door!
I first have to acknowledge that I have not been able to follow the MySQL 5.5 development as closely as I would have wanted during the last 2 years as most of the planning of MySQL 6.0, 5.4 and 5.5 has happened behind closed doors, without insight for the community. The commits have been open, until recently, but it's not easy to follow what is happening just based on the commits. I am sure that I am missing below some of the important features in 5.5 and forgetting to acknowledge some of the people that have done great work on 5.5.
That said, the 5.5 release contains some things that a lot of heavy users of MySQL have waited a long time for. Here follows some of the main things that I know about.
InnoDB improvements:
Some good links for more information about the improvements in 5.5:
- New In MySQL 5.5 Performance_scale Unleashed Presentation
- Introduction to MySQL 5.5
- MySQL 5.5: What's New in Replication
It's now almost exactly 2 years since the MySQL 5.1 GA release, so what's the verdict of 5.5?
The two things that impress me are the work on the InnoDB storage engine and the DLL locking. Most of the other new features are minor (technically) adjustments even if the results, like with the windows speed improvement, can be impressive. As a technical person I have a hard time getting impressed by something that could be done in a few hours and could easily have been done in 5.1.
What is worrying me the most is the things that are not in 5.5. Some if the things that were supposed to be in the next MySQL release are:
Pool of threads (instead of one thread per connection like it is in MySQL 5.1).
Another worry I have is how long Oracle will be able to continue development of MySQL while important core developers continue to leave. Of the above, Konstantin Osipov and Vladislav Vaintroub have already left Oracle and there are not many old-timers left.
When it comes to MariaDB we are continuing to work making MariaDB 5.3 stable. MariaDB 5.1, 5.2 and 5.3 use the InnoDB-plugin compatible XtraDB storage engine, which already has many of the speed improvements of the new InnoDB plugin, but not yet the windows speed improvement.
We have already merged MariaDB 5.3 with MySQL 5.5 and we will release the MariaDB 5.5 tree shortly. It has taken some time as there are a lot of things done in MariaDB 5.3 that conflict with new code in MySQL 5.5 and we also have to add back some of the features removed in MySQL 5.5.
The next big decision is to decide which version we should release next as a stable release; MariaDB 5.3 or MariaDB 5.5.
The problem is that it's very hard to evaluate how stable 5.5 really is as that 5.5 has not been tested by the public enough and some of the bugs in the public bug system, like the critical Bug #33082, can't be accessed. The bug database shows 294 open bugs for MySQL 5.1 and 150 for MySQL 5.5. I haven't however had time to evaluate all public bugs, so I don't know how severe they really are. A lot depends also how many bugs will be filed during the next few weeks after the GA release.
Note: I corrected the above sentence after getting confirmation that the MySQL team has not yet switched to Oracle's internal bug system. This is scheduled to happen first in the spring. Thanks for helping me get the facts right!
The good news is that many of the bugs that I originally reported for the 5.1 GA release have been fixed. However there are still things that could have been better:
I first have to acknowledge that I have not been able to follow the MySQL 5.5 development as closely as I would have wanted during the last 2 years as most of the planning of MySQL 6.0, 5.4 and 5.5 has happened behind closed doors, without insight for the community. The commits have been open, until recently, but it's not easy to follow what is happening just based on the commits. I am sure that I am missing below some of the important features in 5.5 and forgetting to acknowledge some of the people that have done great work on 5.5.
That said, the 5.5 release contains some things that a lot of heavy users of MySQL have waited a long time for. Here follows some of the main things that I know about.
InnoDB improvements:
- The new version 1.1 of the already improved InnoDB plugin is now the default (and only) InnoDB (in 5.1 the InnoDB plugin version 1.0.x was an optional engine).
- Multiple buffer pools and rollback segments, splitting of log_sys and flush_list mutex, and other improvements that together give a major contribution to the improved scalability and performance of 5.5.
- Thanks to the InnoDB team for continuing to do impressive work!
- In earlier MySQL versions LOCK_open, which protected table opening (among other things), was one of the main things that stopped MySQL from scaling on many CPU's, especially when many tables were used. By introducing proper DDL locking this issue is now mostly fixed.
- A big thanks to Konstantin Osipov and Dimitry Lenev for the hard work they have done in getting this to work.
- Thanks to Mikael Ronström for driving this!
- This was actually the result of a minor bug fix for how mutex and condition variables were used on Windows. By using native Vista primitives instead of homegrown ones, a big speedup was achieved.
- Thanks to Vladislav Vaintroub for getting this fixed!
- Thanks to Google and Mark Callaghan's team for providing the initial patches for this.
- Thanks to Marc Alff for driving and developing this.
- Done by MIT student R.J.Silk and Sergei Golubchik for MariaDB 5.2. Donated to Oracle by Monty Program Ab.
Some good links for more information about the improvements in 5.5:
- New In MySQL 5.5 Performance_scale Unleashed Presentation
- Introduction to MySQL 5.5
- MySQL 5.5: What's New in Replication
It's now almost exactly 2 years since the MySQL 5.1 GA release, so what's the verdict of 5.5?
The two things that impress me are the work on the InnoDB storage engine and the DLL locking. Most of the other new features are minor (technically) adjustments even if the results, like with the windows speed improvement, can be impressive. As a technical person I have a hard time getting impressed by something that could be done in a few hours and could easily have been done in 5.1.
What is worrying me the most is the things that are not in 5.5. Some if the things that were supposed to be in the next MySQL release are:
Pool of threads (instead of one thread per connection like it is in MySQL 5.1).
- As far as I know, this feature has been moved to appear in 5.6 enterprise and will not be part of the community server.
- This project was discontinued by Oracle the same day the feature was deemed ready to be pushed into 5.5.
Another worry I have is how long Oracle will be able to continue development of MySQL while important core developers continue to leave. Of the above, Konstantin Osipov and Vladislav Vaintroub have already left Oracle and there are not many old-timers left.
When it comes to MariaDB we are continuing to work making MariaDB 5.3 stable. MariaDB 5.1, 5.2 and 5.3 use the InnoDB-plugin compatible XtraDB storage engine, which already has many of the speed improvements of the new InnoDB plugin, but not yet the windows speed improvement.
We have already merged MariaDB 5.3 with MySQL 5.5 and we will release the MariaDB 5.5 tree shortly. It has taken some time as there are a lot of things done in MariaDB 5.3 that conflict with new code in MySQL 5.5 and we also have to add back some of the features removed in MySQL 5.5.
The next big decision is to decide which version we should release next as a stable release; MariaDB 5.3 or MariaDB 5.5.
The problem is that it's very hard to evaluate how stable 5.5 really is as that 5.5 has not been tested by the public enough and some of the bugs in the public bug system, like the critical Bug #33082, can't be accessed. The bug database shows 294 open bugs for MySQL 5.1 and 150 for MySQL 5.5. I haven't however had time to evaluate all public bugs, so I don't know how severe they really are. A lot depends also how many bugs will be filed during the next few weeks after the GA release.
Note: I corrected the above sentence after getting confirmation that the MySQL team has not yet switched to Oracle's internal bug system. This is scheduled to happen first in the spring. Thanks for helping me get the facts right!
The good news is that many of the bugs that I originally reported for the 5.1 GA release have been fixed. However there are still things that could have been better:
- regression in parser compared to 5.1.x. This was reported just after 5.5 was released, but something like this should have been noticed during testing.
- regression compared to 5.1.x.
- cannot select from innodb_trx when trx_query contains blobs that aren't strings
- Object names not accessible via case sensitive reference on lctn=1 systems
- Change in execution plan for count_distinct_group_on_key causes 400 % performance drop
- cost calculations (in optimizer) are off
- Slowdown (related to logging) in 5.1.21 vs. 5.1.20
- Stored routines do not detect changes in meta-data.
- crash when federated table loses connection during insert ... select
- comparisons with Information schema tables don't honor collation
- Partitioning performance drops drastically with hundreds of partitions
- Event set to DISABLED and ON COMPLETION NOT PRESERVE is deleted at server start
2010-12-14
In search of a BSD/LGPL/Apache licensed client library for MySQL and MariaDB
We have received many requests for a client library for MariaDB (and MySQL) under a license other than the GPL, and decided that it's now time to do something about it.
Some time ago we released the LGPL client library used in MySQL 3.23. This can be used by many applications to connect to MariaDB, MySQL, and Drizzle, but doesn't satisfy all the requirements one would like to see for a true replacement for the current GPL client library.
Minimum requirements are:
We have already found some companies that are willing to sponsor this project, but not yet enough to get it all done.
If you are willing to donate some time and effort on this, please contact us at
'maria-developers at lists.launchpad.com' or 'community at askmonty.org' (we strongly prefer the public maria developers list).
If you are willing to sponsor this effort, please contact us at 'sales at askmonty.org' or use our contact form.
We will put all monetary sponsorship into the Worklog. As with all worklog items, if someone from the community delivers a working solution that is accepted into the MariaDB source they will get 60% of the so far committed money.
Some time ago we released the LGPL client library used in MySQL 3.23. This can be used by many applications to connect to MariaDB, MySQL, and Drizzle, but doesn't satisfy all the requirements one would like to see for a true replacement for the current GPL client library.
Minimum requirements are:
- It should be binary link level compatible with the current MySQL 5.1 client library. To use it, one would just have to re-link the application with the new library. No re-compilation of the source should be necessary.
- It should have all functionality of the MySQL 5.1 client library (including prepared statements).
- It should be able to connect to all current versions of MariaDB, MySQL and Drizzle.
- It should support all the new functionality in MariaDB's client library like new client functions and pluggable authentication.
- The license should be one of BSD/LGPL/Apache.
We have already found some companies that are willing to sponsor this project, but not yet enough to get it all done.
If you are willing to donate some time and effort on this, please contact us at
'maria-developers at lists.launchpad.com' or 'community at askmonty.org' (we strongly prefer the public maria developers list).
If you are willing to sponsor this effort, please contact us at 'sales at askmonty.org' or use our contact form.
We will put all monetary sponsorship into the Worklog. As with all worklog items, if someone from the community delivers a working solution that is accepted into the MariaDB source they will get 60% of the so far committed money.
2010-11-15
MariaDB 5.2 is released as stable
I am happy to announce that MariaDB 5.2.3 is now released as a stable release.
During the gamma period we did not receive any serious reports for issues in 5.2, so we are relatively confident that the new code is of decent quality.
You can read about the features of MariaDB 5.2 in my previous blog entry or in the fast growing MariaDB knowledgebase..
What is most interesting about MariaDB 5.2 is that most of the features came from the MariaDB/MySQL community, not from Monty Program Ab!
Without the community it would not have been possible to do a stable release so soon after the last release. Virtual columns, Extended User Statistics, Segmented MyISAM key cache, Pluggable Authentication, OQGRAPH and the Sphinx client are all from code provided by people outside of Monty Program Ab. Storage-engine-specific CREATE TABLE, Enhancements to INFORMATION SCHEMA.PLUGINS table and "Group commit" for the Aria engine were provided by us at Monty Program Ab.
Thanks also to all those who have reported and provided bug fixes for 5.1 and 5.2!
MariaDB 5.2.3 has all changes from MariaDB 5.1.50 and MySQL 5.1.51. (We are just about to release MariaDB 5.1.51)
Please report any issues to the MariaDB bugs database so that we can fix them!
We will continue to fix critical bugs in MariaDB 5.1, even if the
attention of bug fixes will now move to 5.2.
Now we are busy working on getting MariaDB 5.3 ready for beta. We have also started a merge of MariaDB 5.3 + MySQL 5.5 -> MariaDB 5.5 and hope to release this tree soon!
Happy database usage!
During the gamma period we did not receive any serious reports for issues in 5.2, so we are relatively confident that the new code is of decent quality.
You can read about the features of MariaDB 5.2 in my previous blog entry or in the fast growing MariaDB knowledgebase..
What is most interesting about MariaDB 5.2 is that most of the features came from the MariaDB/MySQL community, not from Monty Program Ab!
Without the community it would not have been possible to do a stable release so soon after the last release. Virtual columns, Extended User Statistics, Segmented MyISAM key cache, Pluggable Authentication, OQGRAPH and the Sphinx client are all from code provided by people outside of Monty Program Ab. Storage-engine-specific CREATE TABLE, Enhancements to INFORMATION SCHEMA.PLUGINS table and "Group commit" for the Aria engine were provided by us at Monty Program Ab.
Thanks also to all those who have reported and provided bug fixes for 5.1 and 5.2!
MariaDB 5.2.3 has all changes from MariaDB 5.1.50 and MySQL 5.1.51. (We are just about to release MariaDB 5.1.51)
Please report any issues to the MariaDB bugs database so that we can fix them!
We will continue to fix critical bugs in MariaDB 5.1, even if the
attention of bug fixes will now move to 5.2.
Now we are busy working on getting MariaDB 5.3 ready for beta. We have also started a merge of MariaDB 5.3 + MySQL 5.5 -> MariaDB 5.5 and hope to release this tree soon!
Happy database usage!
2010-10-15
Greetings from MariaDB meeting in Istanbul
I am now back in Finland after the Monty Program Ab / MariaDB meeting in Istanbul.
As all our meetings, this meeting was also open to the public (except a few closed session about Monty Program Ab internals). As before, we had guests attending, and this time we had people from Facebook, Intel, Galera, Percona, SkySQL and from another database company. PBXT sent their greetings but could not attend because of customer meetings.
During the meetings, we presented our development plans for replication and for the new Knowledgebase system (KB). One of the objectives with the KB is to eventually create a free MariaDB / MySQL manual, but it's also a place to ask questions and find/get answers. We also had some hacking sessions, which gave me time to review the phone home feature that we will add to MariaDB 5.2. We also discussed the dynamic columns feature that is planned for MariaDB 5.3. We also had great presentations from Facebook and Intel that will help us improve MariaDB.
You can find the full Istanbul program here.
It was also very fun to be part of the SkySQL kickoff meeting that also happened to be in Istanbul (We didn't know that SkySQL would also be there when we planned our meeting. The two teams were mostly meeting at separate locations, but sharing one day together.). It was great to see a lot of old friends from the MySQL time and it felt like we had used a time machine to go 5 year back into the past (especially when "Helan går" was sung many days in a row by people that doesn't really speak Swedish). The old MySQL spirit is very much alive!
We had of course also a lot of good food (mostly kebab in a lot of different forms, including raw), visited a hamam, and did some sport and dancing. Henrik Ingo has documented some (un)memorable pieces in his blog.
What is becoming more and more clear is that MariaDB is not only a Monty Program Ab project. MariaDB is not only getting a lot of contributions from individual developers but there is also an increasing collaboration between companies working on the code. MariaDB 5.2 includes for example code (in no particular order) from Percona, OpenQuery, Primebase, Google, Sphinxsearch and Stardoll, and naturally from Oracle. In 5.3 and/or 5.5 we will in addition work with companies like Galera, Facebook, eBay, Sphaera and of course SkySQL to add many new features into MariaDB.
Next Monty Program Ab / MariaDB meeting will probably be in 6-9 months in Lissabon. Hope to see many of you there!
As all our meetings, this meeting was also open to the public (except a few closed session about Monty Program Ab internals). As before, we had guests attending, and this time we had people from Facebook, Intel, Galera, Percona, SkySQL and from another database company. PBXT sent their greetings but could not attend because of customer meetings.
During the meetings, we presented our development plans for replication and for the new Knowledgebase system (KB). One of the objectives with the KB is to eventually create a free MariaDB / MySQL manual, but it's also a place to ask questions and find/get answers. We also had some hacking sessions, which gave me time to review the phone home feature that we will add to MariaDB 5.2. We also discussed the dynamic columns feature that is planned for MariaDB 5.3. We also had great presentations from Facebook and Intel that will help us improve MariaDB.
You can find the full Istanbul program here.
It was also very fun to be part of the SkySQL kickoff meeting that also happened to be in Istanbul (We didn't know that SkySQL would also be there when we planned our meeting. The two teams were mostly meeting at separate locations, but sharing one day together.). It was great to see a lot of old friends from the MySQL time and it felt like we had used a time machine to go 5 year back into the past (especially when "Helan går" was sung many days in a row by people that doesn't really speak Swedish). The old MySQL spirit is very much alive!
We had of course also a lot of good food (mostly kebab in a lot of different forms, including raw), visited a hamam, and did some sport and dancing. Henrik Ingo has documented some (un)memorable pieces in his blog.
What is becoming more and more clear is that MariaDB is not only a Monty Program Ab project. MariaDB is not only getting a lot of contributions from individual developers but there is also an increasing collaboration between companies working on the code. MariaDB 5.2 includes for example code (in no particular order) from Percona, OpenQuery, Primebase, Google, Sphinxsearch and Stardoll, and naturally from Oracle. In 5.3 and/or 5.5 we will in addition work with companies like Galera, Facebook, eBay, Sphaera and of course SkySQL to add many new features into MariaDB.
Next Monty Program Ab / MariaDB meeting will probably be in 6-9 months in Lissabon. Hope to see many of you there!
2010-10-01
MariaDB 5.2.2-gamma is released
MariaDB 5.2.2-gamma is released
MariaDB 5.2 is finally released as gamma (RC). I had hoped to release this in July at Oscon but our new QA person, Philip Stoev, find at the last moment some problems with Aria recovery and virtual columns that we wanted to fix before doing the release.
The new features in 5.2 are quite isolated and as most have been in use by members in the MySQL community for a long time, we don't expect any big problems with 5.2 and we should be able to declare it stable within a few months.
5.2.2 gamma is based on MariaDB/MySQL version 5.1.50. It has the following new features:
We will now continue with working on MariaDB 5.3. As soon as MySQL 5.5 is stable enough, we will create a MariaDB 5.5 tree (based on MariaDB 5.3 + MySQL 5.5) and start working on that.
Please download MariaDB 5.2, test and give us feedback on maria-discuss@lists.launchpad.net or file a bug. We believe it's already quite stable, but we will only know after we have enough users testing it!
MariaDB 5.2 is finally released as gamma (RC). I had hoped to release this in July at Oscon but our new QA person, Philip Stoev, find at the last moment some problems with Aria recovery and virtual columns that we wanted to fix before doing the release.
The new features in 5.2 are quite isolated and as most have been in use by members in the MySQL community for a long time, we don't expect any big problems with 5.2 and we should be able to declare it stable within a few months.
5.2.2 gamma is based on MariaDB/MySQL version 5.1.50. It has the following new features:
- Virtual columns
- Columns that are an expression and are calculated on retrieval.
- Extended User Statistics
- Client, User, Index and Table statistics.
- Segmented MyISAM key cache
- The key cache's global mutex is split into several mutex which gives a notable speed improvement under multi user load; We have registered up to 70% more performance thanks to this.
- Pluggable Authentication
- Authentication is done via an extensible plugin, which makes it easy to add any kind of authentication to MariaDB.
- Storage-engine-specific CREATE TABLE
- Allows one to specify additional attributes per field, index or table to the storage engine.
- Enhancements to INFORMATION SCHEMA.PLUGINS table
- We expose more information about the plugins, like maturity levels.
- Group commit for the Aria engine.
- Speeds up multi user inserts.
- OQGRAPH (experimental quality)
- Allows you to handle hierarchies (tree structures) and complex graphs (nodes having many connections in several directions).
- Sphinx, a text-search engine (experimental quality)
- A built-in Sphinx client which allows MariaDB to talk to searchd, run search queries, and obtain search results.
We will now continue with working on MariaDB 5.3. As soon as MySQL 5.5 is stable enough, we will create a MariaDB 5.5 tree (based on MariaDB 5.3 + MySQL 5.5) and start working on that.
Please download MariaDB 5.2, test and give us feedback on maria-discuss@lists.launchpad.net or file a bug. We believe it's already quite stable, but we will only know after we have enough users testing it!
2010-07-18
What is an Open Source Company?
One of the hot topics here at the Community Leadership Summit in Portland is "what is an open source company ?". Simon Phipps has a got a lot of good points on this in his blog about Open Source Business.
We have companies like SugarCRM and Eucalyptus marketing themselves as "open source companies", even while not all of their code is available under an open source license.
To me it's clear that just because some of your product(s) is available under an open source license, you can't claim to be an open source company, as that would make the term meaningless. Under such a definition even Microsoft would be an open source company, as some of their products are now available as open source.
SugarCRM and Eucalyptus are clearly 'open core' companies, not open source companies. While open core is somewhat better than closed source, open core products have all the same disadvantages as closed source if you depend on a single feature of the closed parts for your business. In this case:
- You can't change, modify, port or redistribute the code.
- You can't fix bugs or extend the code.
- You are locked to the platforms that the vendor provides
- You are locked to one vendor.
In other words, the product as a whole should be regarded as a closed source product.
A little background why I feel so strongly about the term "open source company".
When MySQL AB was founded, David's and my intention was to create an open source company. Our definition was back then very simple "all software we produce should be under an open source license". When we took in investors we ensured that MySQL AB would stay as an open source company by putting a clause about this in our shareholder contracts.
David and I did however make a small mistake in that the shareholder agreement only said that "MySQL software" should be kept under an open source license. This allowed the MySQL management in 2006 to release Merlin, the MySQL monitor, as a closed source product, by claiming "this was not based on the MySQL server code". So even if we, the founders, managed to keep the MySQL server free, MySQL AB was only an "open source company" until 2006.
Learning from my mistake and to ensure that Monty Program Ab would always be an open source company, Zak Greant and I created the Hacking business model. Monty Program Ab follows this model and has additionally made a public promise that everything we create and release to our users will be under an open source license.
So what would then be a good definition for calling onces company "an open source company"?
I would like to suggest the following one:
1) You have to be a company that produces software.
2) All software the company delivers to its users must be available to everyone under an open source license. This includes all server code that is required to run and use the software.
In addition it would be good if the company could publicly state that all code they produce and release in the future will be under an open source license, but personally I would not require the company's to have to do this as some companies would have a hard time to do this.
At least here at the Leadership summit, the above definition seems to be acceptable to those that I have talked to. Please comment what you think about this!
We have companies like SugarCRM and Eucalyptus marketing themselves as "open source companies", even while not all of their code is available under an open source license.
To me it's clear that just because some of your product(s) is available under an open source license, you can't claim to be an open source company, as that would make the term meaningless. Under such a definition even Microsoft would be an open source company, as some of their products are now available as open source.
SugarCRM and Eucalyptus are clearly 'open core' companies, not open source companies. While open core is somewhat better than closed source, open core products have all the same disadvantages as closed source if you depend on a single feature of the closed parts for your business. In this case:
- You can't change, modify, port or redistribute the code.
- You can't fix bugs or extend the code.
- You are locked to the platforms that the vendor provides
- You are locked to one vendor.
In other words, the product as a whole should be regarded as a closed source product.
A little background why I feel so strongly about the term "open source company".
When MySQL AB was founded, David's and my intention was to create an open source company. Our definition was back then very simple "all software we produce should be under an open source license". When we took in investors we ensured that MySQL AB would stay as an open source company by putting a clause about this in our shareholder contracts.
David and I did however make a small mistake in that the shareholder agreement only said that "MySQL software" should be kept under an open source license. This allowed the MySQL management in 2006 to release Merlin, the MySQL monitor, as a closed source product, by claiming "this was not based on the MySQL server code". So even if we, the founders, managed to keep the MySQL server free, MySQL AB was only an "open source company" until 2006.
Learning from my mistake and to ensure that Monty Program Ab would always be an open source company, Zak Greant and I created the Hacking business model. Monty Program Ab follows this model and has additionally made a public promise that everything we create and release to our users will be under an open source license.
So what would then be a good definition for calling onces company "an open source company"?
I would like to suggest the following one:
1) You have to be a company that produces software.
2) All software the company delivers to its users must be available to everyone under an open source license. This includes all server code that is required to run and use the software.
In addition it would be good if the company could publicly state that all code they produce and release in the future will be under an open source license, but personally I would not require the company's to have to do this as some companies would have a hard time to do this.
At least here at the Leadership summit, the above definition seems to be acceptable to those that I have talked to. Please comment what you think about this!
Subscribe to:
Posts (Atom)
Posts
