MariaDB Foundation achivements 2012-12 - 2013-09

I recently read some comments that we at the MariaDB Foundation have not been very open about what we are doing.

We are very sorry about this. The problem is not that we are secret about what we are doing, the problem is that not many of us working at the MariaDB Foundation are very active bloggers.

I will try to address this concern by starting a monthly blog about the MariaDB development that MariaDB Foundation employees are doing. This together with Simon Phipps' state of the sea lion blog, which is published on the MariaDB Foundation blog, should hopefully give everyone a better idea of what we are doing.

At the MariaDB Foundation we are now 4 people working full time on code, documentation, and spreading MariaDB. Here are the things we have done since the Foundation was formed in December 2012:

Major development efforts based on code from the community:

• MDEV-4568 Port Percona response time distribution as audit plugin
• MDEV-4911 Add KILL query id, and add query id information to process list
• MDEV-4684 Enhancement request: --init-command support for mysqlslap

Major new development efforts:

• MDEV-4702 Reduce usage of LOCK_open
• MDEV-4011 Per thread memory usage
• MDEV-4902 sql_yacc.yy incompatible with bison 3
• MDEV-4950 mysql_upgrade fails with disabled InnoDB (a bug that is not fixed in MySQL)
• MDEV-21 LGPL C client library
• LGPL Java client library.
• Some work on a new LGPL ODBC driver for MySQL and MariaDB.
• Better error messages (the error code in an error message now has descriptive text)
• Faster connect (lots of small fixes).
• MDEV-3941 CREATE TABLE xxx IF NOT EXISTS should not block if table exists.

The MariaDB Foundation has also done a lot of work on the merge of MySQL 5.6 to MariaDB 10.0. The most notable features done by the Foundation are:

• All InnoDB changes
• All performance schema changes
• Cleaned up the HANDLER interfaces (to make life easier for storage engine writers).
• TRANSACTION READ ONLY | READ WRITE
• MySQL-5.6 TIME/TIMESTAMP/DATETIME data types with fractional seconds
• MySQL-5.6 temporal literals
• utf16le character set
• TO_BASE64 and FROM_BASE64 SQL functions
• MySQL-5.6 collation customization improvements (in progress)

Note that the above is not a complete list of what is in MariaDB 10.0!


We are working with external entities and contributors to get features into MariaDB. Some of these have required a lot of work on the MariaDB Foundation side:

• CONNECT storage engine
  • Porting to various operating systems (*BSD, Linux flavors, Solaris)
  • Porting to unixODBC
  • Fixing compiler warnings
  • Testing and writing mtr tests
  • Checking, reporting and fixing valgrind warnings
  • Reviewing the author's code
  • Adding MySQL/MariaDB security support (FILE privilege and secure_file_priv)
  • Adding ConnectSE support in embedded server
  • Globalization: making ConnectSE support multiple MariaDB character sets
  • Discussing many aspects of better integration of ConnectSE into MariaDB with the author.
  • Packaging (RPMs, DEBs, MSI)
• TokuDB storage engine
  • Review of code
  • Some small integration changes.
• Parallel slave
• Multi source replication
  • Worked closely with the original author to extend the code from a working proof of concept to production level code.
• MDEV-4438 Spider storage engine
  • Continuously working with Kentoku
• MDEV-4425 REGEXP enhancements (a Google Summer of Code project, in progress)
  • Mentoring Sudheera Palihakkara (the student)
  • Student's code reviews (for RLIKE and RLIKE_REPLACE)
  • Code enhancements proposals
  • CMake related enhancements
  • Wrote RLIKE_SUBSTR and RLIKE_INSTR functions
• Added support for --crash-script in mysqld_safe.

A lot of basic system maintenence:

• Reported a lot of bugs (this is not a full list):
  • MDEV-4489 Replication of big5, cp932, gbk, sjis strings makes wrong values on slave
  • MDEV-4634 Crash in CONVERT_TZ
  • MDEV-4635 Crash in UNIX_TIMESTAMP(STR_TO_DATE('2020','%Y'))
  • MDEV-4651 Crash in my_decimal2decimal in a ORDER BY query
  • MDEV-4652 Wrong result for CONCAT(GREATEST(TIME('00:00:01'),TIME('00:00:00'))
  • MDEV-4653 Wrong result for CONVERT_TZ(TIME('00:00:00'),'+00:00','+7:5')
  • MDEV-4654 Wrong warning for CAST(TIME('10:20:30') AS DATE) + INTERVAL 1 DAY
  • MDEV-4655 Difference in how GREATEST and COALESCE process arguments
  • MDEV-4714 Wrong TIME value from a long string
  • MDEV-4724 MAKETIME does not preserve microseconds
  • MDEV-4838 Wrong metadata for DATE_ADD('string', INVERVAL)
  • MDEV-4841 "Wrong character set of ADDTIME() and DATE_ADD()"
  • MDEV-4842 STR_TO_DATE does not work with UCS2/UTF16/UTF32
  • MDEV-4843 Wrong data type for TIMESTAMP('2001-01-01','10:10:10')
  • MDEV-4844 Redundant warning in SELECT TIME('00:00:00.')
  • MDEV-4848 Wrong metadata or column type for LEAST(1.0,'10')
  • MDEV-4850 MATCH uses a wrong fulltext index with mismatching collation
  • MDEV-4857 Wrong result of HOUR('1 00:00:00')
  • MDEV-4858 Wrong error message for a huge unsigned value inserted into a TIME column
  • MDEV-4859 Wrong value and data type of "SELECT MAX(time_column) + 1 FROM t1"
  • MDEV-4861 TIME/DATETIME arithmetics does not preserve INTERVAL precision
  • MDEV-4862 Wrong result of MAKETIME(0, 0, 59.9)
  • MDEV-4863 COALESCE(time) returns wrong results in numeric context
  • MDEV-4868 Length of CURRENT_TIME is too long
  • MDEV-4869 Wrong result of MAKETIME(0, 0, -0.1)
  • MDEV-4870 Wrong values of CASE, COALESCE, IFNULL on a combination of different temporal types
  • MDEV-4871 Temporal literals do not accept nanoseconds
  • MDEV-4872 Inserting a huge number into a TIME column produces a wrong result
  • MDEV-4898 Too many warnings when inserting a bad value into a TIMESTAMP column
  • MDEV-4900 Bad value inserted into a TIME field on truncation
• Fixed a LOT of bugs (the following is not a full list):
  • MDEV-4489 Replication of big5, cp932, gbk, sjis strings makes wrong values on slave
  • MDEV-4512 Valgrind warnings in my_long10_to_str_8bit on INTERVAL and DATE_ADD with incorrect types
  • MDEV-4518 Server crashes in is_white_space when it's run with query cache...
  • MDEV-4634 Crash in CONVERT_TZ
  • MDEV-4635 Crash in UNIX_TIMESTAMP(STR_TO_DATE('2020','%Y'))
  • MDEV-4639 my_datetime_to_str: Conditional jump or move depends on uninitialized value
  • MDEV-4651 Crash in my_decimal2decimal in a ORDER BY query
  • MDEV-4652 Wrong result for CONCAT(GREATEST(TIME('00:00:01'),TIME('00:00:00'))
  • MDEV-4653 Wrong result for CONVERT_TZ(TIME('00:00:00'),'+00:00','+7:5')
  • MDEV-4724 MAKETIME does not preserve microseconds
  • MDEV-4804 Date comparing false result
  • MDEV-4819 Upgrade from MySQL 5.6 does not work
  • MDEV-4841 "Wrong character set of ADDTIME() and DATE_ADD()"
  • MDEV-4843 Wrong data type for TIMESTAMP('2001-01-01','10:10:10')
  • MDEV-4863 COALESCE(time) returns wrong results in numeric context
  • MDEV-4871 Temporal literals do not accept nanoseconds
  • Fixed MySQL bug #69861 LAST_INSERT_ID is replicated incorrectly if replication filters are used.
  • Fixed issues with partitions and create temporary table SELECT
  • If one declared several continue handler for the same condition on different level of stored procedures, all of them where executed. Now we only execute the innermost of them (the most relevant).
  • Don't abort InnoDB if one can't allocate resources for AIO
  • Added some fixes that should make MyISAM & Aria REPAIR work with more than 4G records.
  • MDEV-4352 LOAD DATA was not multi-source safe
  • MDEV-4394 Sporadic failures in multi_source tests
  • MDEV-4033 Unable to use slave's temporary directory /tmp - Can't create/write to file '/tmp/SQL_LOAD-' (Errcode: 17 "File exists")
  • Increase default value of max_binlog_cache_size and max_binlog_stmt_cache_size to ulonglong_max.
  • MDEV-4319 mysqlbinlog output ambiguous escaping (patch by Ian Good)
  • OPTION is now a valid identifier (not a reserved word)
  • Fixed issue with LOCK TABLE + ALTER TABLE ENABLE KEYS + SHOW commands.
  • MDEV-628 Querying myisam table metadata may corrupt the table.
  • Replace with an auto_increment primary key and another unique key didn't replicate correctly with REPLACE.
  • BUG#51763 Can't delete rows from MEMORY table with HASH key.
  • MDEV-389 Server crash inserting record on a temporary table after truncating it.
  • Fixed crashing bug in GROUP_CONCAT with ROLLUP
  • MDEV-4013 Password length in replication setup
  • MDEV-4009 main.delayed sporadically fails with "query 'REPLACE DELAYED t1 VALUES (5)' failed: 1317: Query execution was interrupted"
  • Fixed CREATE TABLE IF EXIST generates warnings instead of errors
  • Removed lock wait timeout warning when using CREATE TABLE IF EXISTS
• Code cleanups and code rewrites.
• Continued work to ensure that MariaDB works on many operating systems.
• Did lots of small fixes for Solaris.
• Review of code from the community.
• Removed valgrind and compiler warnings.
• Added new test cases.
• QA & benchmarking.

Talked about MariaDB and Open Source/Free Software at a lot of conferences:

• Percona Live London 2012
• SkySQL Roadshow, Stuttgart
• Vista developer Conference, Riga
• PHP Conference, Pasila, Finland
• SkySQL Roadshow, Amsterdam
• Percona Live, Santa Clara
• SkySQL Roadshow, London
• MySQL Meetup, NYC
• Suomen Arkistointi Seminaari, Mikkeli, Finland
• SkySQL Roadshow, Berlin
• Ali Baba Conference, Hangzhou, China
• OSCON, Portland
• Debconf, Switzerland
• Froscon, St Augustin, Germany
• Haaga-Helia Talks, Finland
• IDCEE Conference, Ukraina
• Latinux, Brazil & Peru
• Highload Conference, Moscow
• Foscon, Gotenburg, Sweden
• Percona Live 2013, London

Other things people from the MariaDB foundation have done:

• Created +200 and updated +3000 articles in the MariaDB Knowledge Base. Some of the notable things are:
• Complete documentation of all system variables
• Complete documentation of all status variables
• Clients and utilities
• Geographic functions
• Performance schema
• Log files
• Internationalization, character sets, collation
• XtraDB and InnoDB content
• Global Transaction ID (from Kristian's content)
• Triggers
• Stored Functions
• Stored Procedures
• Views
• Dynamic Columns
• Fulltext indexes
• Replication
• Date and time
• Updating fill_help_tables.sql so that MariaDB HELP output will point
• the Knowledge Base
• Reformatted most articles for improved readability
• Replaced many of the external links with links to newly created internal articles
• Visited a lot of companies to talk about the MariaDB Foundation.
• Worked closely with OS distributions to get MariaDB accepted as a standard part of their distributions. Partly thanks to the MariaDB Foundation, the following distributions now have MariaDB packages:
  • GNU/Linux KDu
  • Mageia
  • Gentoo
  • openSUSE
  • Slackware
  • Arch Linux
  • ALTLinux
  • MariaDB Debian Live Images
  • Parabola GNU/Linux
  • TurboLinux
  • The Chakra Project
  • Fedora
  • Red Hat Enterprise Linux
  • FreeBSD
  • OpenBSD
  • Debian now has MariaDB packages in 'unstable'.
  • For a full list and to see which distributions are replacing MySQL with MariaDB, see this page.
• Updated SHOW AUTHORS and SHOW CONTRIBUTORS to give more merit to active contributors and sponsors.

We are very thankfull to the MariaDB Foundation members and supporters that have made this work possible!

The current members are:

• Booking.com
• SkySQL
• Parallels

If you want the MariaDB foundation to do even more work like the above, you should ensure that your company becomes member in the MariaDB Foundation! You can also sponsor the foundation as a private person!

MariaDB and MySQL room at Froscon

Froscon, the German Free Software and Open Source conference, is happening again this weekend in University of Applied Sciences Bonn-Rhein-Sieg.

This year we have a MariaDB/MySQL room, C 217, where we can meet, discuss and present MariaDB and MySQL related issues.

I will probably be in the room or at the MariaDB booth most of the time. I welcome people to come to the room with their database problems so that we can try to solve them 'live' (assuming that the Internet is reliable enough).

I will also hold a keynote, How to make money from Open Source today. This includes some information about Business source, a topic that has been discussed in many places lately

If you are looking for information about MariaDB, MySQL, how to make money with Open Source or just have a craving for black vodka, seek me out at Froscon!

Business Source (A software license with some Open Source aspects)

A couple of weeks ago I was interviewed on ZDNET about how to create successful software company in todays world.

I assume that because the original article also mentioned my other project, MariaDB, some people jumped to the wrong conclusion about my intentions or what I was trying to achieve.

For those that want to know more about Business Source, there is now an academic article in TIM (Technology Innovation Management Review) that one can read. The article is written by Linus Nyman and me.

To clarify some misunderstandings, here is a short list of what the Business Source license is all about:

• Business Source is not an Open Source license. It's a commercial software license that offers the users many of the benefits of an Open Source license.
• Business Source means that all source code is available from day one and that most (but not all) users can use it any way for free. After a time delay the software becomes Open Source.
• Business Source was never intended for the MariaDB server. MariaDB is GPL and will always be that. I could not change the license for the MariaDB server even if I wanted to (and I never wanted to make MySQL or MariaDB closed source or Open core).
• I truly belive that Open Source is a better way to develop software. However as an entrepreneur I recognize that it's very hard to create a software development company around Open Source. Dual licenses works for some cases (especially for infrastructure and embeddable software like MySQL), but doesn't work for a lot of software. It's also very hard to fund development with services (support, development, consulting...) as the profit from these (typically 30%) are not enough to maintain a development organization in many cases.
• For a software company that wants to do development and compete with closed source companies on similar economic terms, Business Source offers a viable alternative to closed source or open core. For a user of the software, Business Source is always better than Open Core as Open Core doesn't offer for it's user ANY of the advantages of Open Source.
• While searching for companies that my investment company, Open Ocean Capital, can invest into, I have been talking with a lot of entrepreneurs about how to make money on their software. In some cases where the entrepreneurs would like to release their software as Open Source but don't know how to make enough money to support development, I have told them to take a look at Business Source. As an Open Source/Free Software advocate it feels a bit bad to have to turn away some companies from Open Source, but I think that in the long run it's better that the companies succeed and produces good software that is accessible to anyone under reasonable terms. Compared to other license models, at least with Business Source the software will eventually become Open Source.

The article should hopefully explain any other questions you have about Business Source. If not, feel free to comment on this blog and I will do my best to clarify things.

By the way, I am not totally happy with the term "Business Source" as there is already a company that uses this name. Business Source (or Source for Business) is however the best term I have come up with for this license so far. If you have a better suggestion for the name, please write me a comment!

MariaDB Foundation at the Percona conference

The MariaDB Foundation have just issued press release about the new Governance in the Foundation.  A lot of the new things that is happening in the MariaDB adoption comes thanks to the work we have done in the Foundation.

The Foundation is also happy to announce that is has now 2 senior MySQL (now MariaDB) developers on board, Alexander Barkov and Sergey Vojtovich and a documentation writer, Ian Gilfillan.

The foundation is also helping founding the new Connect engine, which allows you to use MariaDB with a lot of different formats (XML, CVS, DBF,...), and connections, including ODBC. (Documentation will appear shortly here).

If you want to know more about the MariaDB Foundation, you can find me and a lot of MariaDB developers in the MariaDB foundation boot at Percona conference in Santa Clara.

Monty Program Ab, SkySQL and Antony Curtis has also a lot of talks at Percona live about MariaDB and related things:

• MariaDB 10.0 & what's new with the project
• MariaDB Cassandra Interoperability.
• Replication changes in MariaDB.
• Engine-independent persistent statistics with histograms in MariaDB.
• Spatial functions in MySQL 5.6, MariaDB 5.5, PostGIS 2.0 and others.
• Perl Stored Procedures for MariaDB.
• Deploying a highly available database solution in the Amazon EC2 Cloud using MariaDB.
• MHA: Getting started and moving past the quirks.
• Storage Engines and other Plugins: What's New?.

We have also a MariaDB BOF at 6:00 pm on the 23'th of April in Ballroom F. Anyone who ever been on one of the MySQL/MariaDB BOF's with me knows what to expect. This time there will be even more surprises...

Hope to see a lot of you next week!

MariaDB-Galera 5.5 released as stable

It's now about one year ago since we released MariaDB 5.5. That proved to be an important release for MariaDB as it became popular with the users and ultimately also has been adopted by several important Linux distributions. But we have not stopped working since then, and now the MariaDB project is happy to announce something new again: the immediate availability of MariaDB Galera Cluster 5.5.29 stable release (GA).

As soon as we had released MariaDB 5.5 stable, we started planning with Codership Oy how to integrate their Galera replication technology with MariaDB 5.5. We have then worked together to merge their Write Set REPlication API into MariaDB, and gone through a rigorous testing program with beta and RC releases leading to today's a stable release.

Elena from Monty Program and people from the MariaDB user community have tested those releases and provided feedback if they have found any bugs. Seppo Jaakola from the Galera team is a MariaDB Captain, which means he not only has commit rights to the MariaDB trunk, he also can participate in the MariaDB decision making together with other core contributors. (Note that MariaDB Galera Cluster currently still has it's own branch in the MariaDB project on Launchpad. But I think it is likely that in the future this will become part of the main MariaDB branch and releases.)

This is one reason today's release makes me very happy, because this is a great example of what we always wanted the MariaDB project to become. It also shows how we operate with many other companies and individuals. We want to be open and inclusive to anyone who can contribute great code, so that MariaDB can continue its MySQL heritage of being the most popular open source database. We want MariaDB to always include all the great innovation happening in the MySQL ecosystem.

About synchronous multi-master clustering

In the past weeks I have traveled in Germany, Korea and Sweden to speak about MariaDB, and it is clear to me that a lot of people have already heard of Galera and are already trying it out. But if you didn't yet know about this technology, let me tell you why it is important.

MariaDB Galera Cluster provides synchronous multi-master replication. A simple way to explain what this means is to compare it with other alternatives that we have used for MariaDB high availability until today:

If you compare it to the traditional master-slave replication, it means that your data is safer in a Galera cluster because it is replicated immediately as part of the commit, without any delay. (This is why it is synchronous.) Also, in traditional master-slave replication you can do read-only scale-out, but with Galera you can read and write to any node. This makes life easier for application developers, because you don't need to separate read-only transactions and write transactions. (This is why it is called multi-master.)

You can also compare it with DRBD, which is another popular High Availability solution for MariaDB and MySQL. The reason people use DRBD is usually because it is also synchronous replication, only it happens on the disk driver level. So it is used by people who want to be 100% sure they don't lose a single transaction if they do a failover. But DRBD doesn't give you any scale-out, since the spare node is so called cold standby, so you cannot use it for anything else but disaster recovery.

With MariaDB Galera Cluster we now provide "best of both worlds": it is both synchronous, and you can use it for scale-out. And not just read-only scale out but multi-master scale-out.

Automatic node provisioning

There are many other benefits to Galera too, and you will be able to read more about them on the MariaDB blog. One nice little feature I think is also worth mentioning is the automatic node provisioning. One reason MySQL master-slave replication has become so popular is that it is quite easy to setup and understand. But Galera takes this even further, they completely automate the node provisioning process. My philosophy when I created MySQL was always to make everything very easy to the user, and I'm happy to see Galera shares this philosophy.

But it is not only nice, it is actually an important feature especially if you run MariaDB in the cloud. In the cloud you can save money by adding nodes to a cluster (scale out) when needed and then removing them (scale in) when you need less performance. You might do this every day, or every week depending on what kind of traffic your website gets. But to add and remove MariaDB nodes every day, you of course cannot do it manually every time.

For example many NoSQL systems talk about automatic node provisioning in their marketing so that they will sound "cloud compatible". But MariaDB Galera Cluster does it too, so there is no reason to abandon SQL just to have automatic node provisioning.

Commercial support

When you have tested MariaDB Galera Cluster and decided that you like it, and want to run it in production, you of course want to make sure you have proper support in place. This is also important so that the developers can continue to work on the project and make it even better. Codership operates with a similar model like my company Monty Program, they develop the technology and do bug fixes, and they partner with SkySQL and other companies that support MariaDB in order to provide a complete and seamless support experience. You can contact SkySQL to discuss commercial support for both MariaDB and MariaDB-Galera.

You can also support MariaDB Galera Cluster development directly by donating via the MariaDB Foundation. If you specifically want to support Galera development, you should target your donation to "MariaDB Galera Cluster".

MariaDB living in interesting times

An old Chinese curse says "May you live in interesting times".
Fortunately this can be taken both as a curse or as a blessing.

For MariaDB this has definitely been a blessing the last few months.

Since we announced the MariaDB foundation in December 2012 a lot has happened:

• Wikipedia is moving to MariaDB.
• Mozilla is also now using MariaDB.
• Fedora and OpenSuse is replacing MySQL with MariaDB in their distributions.
• Chakra is also moving to MariaDB.
• We have released LGPL version of the MariaDB/MySQL C and Java Connectors.

We will soon be able to announce that also some of the other very big Internet companies have moved to MariaDB. (stay tuned :)

The above means that the installation base of MariaDB will increase from about one million to 5-10 million within this year. You can find here a full updated lists of all distributions that supports MariaDB.

We are really living in interesting times!

Happy Chinese New Year to you all!