2014-10-01

Why SkySQL becoming MariaDB Corporation will be good for the MariaDB Foundation

Today SkySQL is changing its name to MariaDB Corporation. This is something that I had both anticipated and I think it's a great step for MariaDB.

I wanted here to to share my thoughts on how this change affects the MariaDB community.

The short version: As the MariaDB Corporation is the main driving force behind the development of the MariaDB server and the biggest support provider for it, it makes sense to give it a name that clearly communicates this fact.  The name change doesn't of course stop the company to continue it's excellent support for MySQL.

For MariaDB users and customers, the name change should not affect them in any way, except that it will make it easier for them to find more information about MariaDB as there is fewer names involved.

For the MariaDB Foundation, there is no big change either. After all, the MariaDB foundation was created to protect the MariaDB server, not the MariaDB trademark as such.

The longer version, for those who want more context, starts with some history.

After the Sun acquisition of MySQL AB, I started Monty Program to work on a branch of the MySQL code base named MariaDB after my youngest daughter. In 2010, I was one of the founders behind SkySQL as an alternative service provider for Oracle MySQL customers. Last year SkySQL merged with Monty Program to increase the support behind MariaDB. As the adoption of MariaDB has grown, SkySQL’s business has evolved to provide products and services to over 2 million MariaDB users.

Talking about a company called SkySQL, that provides subscription services for MariaDB while also supporting MySQL, was becoming too complicated and confusing. To make things simpler, and reinforce the company’s focus, I both agreed and recommended that a name change was due.

Having the company using the MariaDB name will also help ensure that the company will focus on MariaDB and put even more development resources on MariaDB.

I assume that some people will wonder if the MariaDB Foundation is needed anymore?  I think it's needed now more than ever to ensure that the MariaDB server is always guaranteed to be open for development by the community! The Foundation will continue in its role at the center of the open, independent and dynamic community that drives the adoption of MariaDB.  The Foundation will also need more paying members to be able to continue interacting with the ever growing external MariaDB developer community.

We’ve been working with the team at MariaDB Corporation (formerly SkySQL!) and have come to agreement on how the trademark will be used. Details of this will be published soon on http://www.mariadb.org.

I continue to believe passionately that the world needs an open, actively developed relational database platform that is adopting to your needs and is better suited for modern web scale application development than other alternatives. MariaDB is that platform. We, the MariaDB developers and all other people at the MariaDB foundation and MariaDB Corporation, are all excited that so many of you are choosing MariaDB. We are all committed to making this choice a success.

MariaDB would not be what it is today without your support!  Thank you for this!

2014-08-05

Logs and more logs, who has time to read them ?

While working on some new features in MariaDB 10.1, I noticed that a normal user couldn't disable the slow query log, which I thought was a bit silly.

While implementing and documenting this feature, I noticed that the information about the different logs is quite spread around and it's not that trivial to find out how to enable/disable the different logs.

To solve this, I created a new MariaDB kb entry, "Overview of the MariaDB logs that I hope MariaDB and MySQL users will be find useful.

Here follows a copy of the kb entry. If you have any comments or things that could be added, please do it in the kb entry so that it will benefit as many as possible!

Overview of MariaDB logs

There are many variables in MariaDB that you can use to define what to log and when to log.

This article will give you an overview of the different logs and how to enable/disable logging to these.

The error log

  • Always enabled
  • Usually a file in the data directory, but some distributions may move this to other locations.
  • All critical errors are logged here.
  • One can get warnings to be logged by setting log_warnings.
  • With the mysqld_safe --syslog option one can duplicate the messages to the system's syslog.

General query log

  • Enabled with --general-log
  • Logs all queries to a file or table.
  • Useful for debugging or auditing queries.
  • The super user can disable logging to it for a connection by setting SQL_LOG_OFF to 1.

Slow Query log

The binary log

Examples

If you know that your next query will be slow and you don't want to log it in the slow query log, do:

SET LOCAL SLOW_QUERY_LOG=0;

If you are a super user running a log batch job that you don't want to have logged (for example mysqldump), do:

SET LOCAL SQL_LOG_OFF=1, LOCAL SLOW_QUERY_LOG=0;

mysqldump in MariaDB 10.1 will do this automatically if you run it with the --disable-log-querys option.

See also

2014-05-20

For your eyes only (or Adding better encryption to MariaDB)

With MariaDB and MySQL we have always taken security seriously.

In MariaDB 10.0 we added roles to make it easier to administrate many users.

MariaDB and MySQL has also many different encryption functions, but what has been neglected in the past is to make encryption easy to use.

This is now about to change.

I recently had a meeting with Elmar Eperiesi-Beck from eperi about simplifying the usage of encryption. We agreed to start a close collaboration around encryption for MariaDB with an agenda to deliver something very secure and easy to use soon.

The things we are initially focusing on are:

  • Adding column level encryption.
    • This will be done at the field level, invisible for the storage engine.
  • Block level encryption for certain storage engines.
    • Initially we will target InnoDB and XtraDB.

MariaDB will initially support storing the security keys on a remote file systems, accessed only at startup, and later also support using a daemon for key management.

The above will make your encrypted data in MariaDB secure for:

  • Database users that has user access to the database.
  • Anyone that would attempt to steal the hard disk with the database.

By using the daemon approach a MariaDB installation will even be secure against database administrators, as they will not have any way to access the key data.

eperi has 11 years of experience with encryption and I am very happy to see them engage with MariaDB to provide better security to MariaDB users!

2014-04-25

conference, conferences...

It's now 3 weeks since the MariaDB & MySQL community day in Santa Clara.

Thanks everyone for coming!

Personally I think it was a success, especially considering the short
time we had to put it together! 11 great speakers and 100+ participants.

We had a small issue with the camera that we used to record all talks: The slides are not very visible. We have been working on editing the videos for all talks to fix this and will update the conference page with both slides and videos for the talks as soon as the editing is finished. The first video is already available! Hope you like it!

We plan to have another MariaDB & MySQL community day in mid November in Florida and another one in Europa after the summer.

Please contact me at 'foundation 'at' mariadb (dot) org' if you want to participate in any of these events!

For the Santa Clara community day we didn't have time to involve the community in selecting the speakers. For the next community days we will work openly with the MariaDB community to select the speakers and plan the event!

I am now attending the LinuxFest Northwest conference where I have a talk about "MariaDB 10.0", which is now declared stable, and "How to make money with open source". Look me up if you want to talk with me about these topics or if you want to discuss, sponsor, or be part of developing any of the features we plan for MariaDB 10.1.

2014-03-20

Scheduled talks for the MariaDB & MySQL community event in Santa Clara


We have now a great set of talks for the MariaDB & MySQL community event in Santa Clara on 3rd of April!

You can find the current scheduled talks here.

Initially we had a few additional talks by other community members, who however had to cancel because of contractual reasons with Percona Live.

We can still fit in a few extra talks by adjusting the schedule. If you want to present something that you think is important for most of the MariaDB and MySQL community, please connect with us at 'foundation' 'at' mariadb (dot) org' or add a comment to this blog.

This is going to be the best event this year if you want to know more about MariaDB and what is happening around MariaDB and MySQL!

You will not only be able to attend great talks, you will also get to talk directly with many of the original creators of MariaDB and MySQL!

Don't worry if you happen to miss some of the talks. We plan to put all talks on YouTube, so that you can view them later at your convenience.

Because of the rush of setting up this conference we did not have time to have a proper community board choose and review the talks. We plan to fix this for the next MariaDB & MySQL community event. The vision is to organize 2-4 free community events per year where all companies in the MariaDB and MySQL community can participate on equal terms.

We are thinking about having the next MariaDB foundation conference in Europa and the following one on the USA east coast. These will be standalone events later this year.

Please contact me if you want to be part of organizing or participate in these or future events

2014-03-05

MariaDB & MySQL community event 2014 in Santa Clara

I am happy to announce that the MariaDB Foundation is organising a MariaDB & MySQL community event in Santa Clara on Thursday the 3rd of April. The venue is the Hilton Santa Clara hotel, a short walk from the Percona Live 2014 event.

The community event is hosted by the MariaDB Foundation with support from AccelerationDB. This is a free community event to complement the Percona Live event. The community event will be a full day focusing on many things that are not presented at Percona Live.

If you are coming to the community event, why not also go to the expo hall ($75) in the convention center as well and support all the vendors there.

We were partly inspired to do this by Baron Schwartz blog post announcing the Percona Performance conference in 2009. We believe that there should be more free conferences about MariaDB and MySQL that will allow anyone to participate. Personally I would also like to see more conferences where the speakers are drawn from all the people that create and continue to innovate in the technology, rather than conferences where a majority of the speakers come from a single company.

The themes for this community event are MariaDB 10.0 GA, High availability and Performance.

In the next MariaDB & MySQL community event we plan to also host a MariaDB and MySQL bootcamp. We where not able to do it this year because of lack of funding and time (if anyone would like to help us do it this time, please contact me!).

We already have a lot of proposed talks from MariaDB developers, Galera developers and some other active community members.
Topics include:
  • MariaDB 10.0 GA, the new features
  • Spider, storage engine with built in sharding
  • Connect, storage engine that allows you to talk with the world (Oracle, PostgreSQL, files etc...)
  • Galera overview and case studies
  • Show case how to insert continuously 1M rows/seconds while doing concurrent reads with MariaDB and ScaleDB
  • How we optimized MariaDB; True case studies from the programmers vault
  • MySQL MHA and Continuent Tungsten shootout 2.0
  • MariaDB multi source replication capability
  • Scaling MySQL (case study)
  • Using ROLES to get more security
We are still looking for more speakers from different companies to make this the best free and community driven MariaDB and MYSQL event in 2014! If you want to talk at the community event, please send an email to 'foundation 'at' mariadb (dot) org'.

We will also organise a dinner that will happen on the same Thursday at the Taste restaurant, a very cool place right around the corner from Birks and Pedros. As the event is free, you will need to pay for the food but we hope to get some further sponsorships for some free drinks (in addition to the inevitable black vodka).

You can register to attend the conference and/or dinner here.

You can use the 'foundation 'at' mariadb (dot) org' email address if you want to sponsor the community event. As MariaDB foundation is a non profit organisation, all sponsorships will go to pay for the event venue, hotel and travel for speakers (who could not otherwise afford to attend); in the event there's anything left over the Foundation will use it for further community activities.

2014-02-10

The final piece of the puzzle

I just pushed the new CREATE OR REPLACE TABLE syntax into MariaDB 10.0, for the soon to be released 10.0.8-gamma (RC). (Before we had only CREATE OR REPLACE for views)

When using the new syntax, the CREATE statement will automatically DROP the old table if it existed.

This is the last feature (which is also a bug fix) depending on me that needed to be pushed before we could release 10.0 gamma (RC). Next, I will start working on speed optimizations and features in 10.1.

The CREATE OR REPLACE TABLE syntax was needed to make global transaction id (GTID) work reliably with CREATE ... SELECT, both in statement-based and row-based replication.

We (Kristian Nielsen and I) didn't think that the solution used in MySQL 5.6 (to give an error message "CREATE TABLE ... SELECT is forbidden when @@GLOBAL.ENFORCE_GTID_CONSISTENCY = 1") when using CREATE ... SELECT was good enough. We wanted something better.

The solution now implemented ensures that we can store DROP TABLE + CREATE TABLE + INSERT INTO TABLE under one GTID.  The GTID entry can also be re-executed in case of slave failure during execution.

While developing CREATE OR REPLACE, I noticed several possible problems in the replication code that were not properly taken care of (neither in MySQL or MariaDB):
  • Having different storage engines on master and slave for any table would not work well together with the GTID code and would cause inconsistencies between GTID's in the master and slave.
  • If CREATE SELECT would fail on the slave, there was no way the slave could continue as it could not roll back the CREATE statement.
  • Slave failure during a DROP TABLE would make the slave stop and it would be unable to restart without user intervention.
  • Having different replication modes on master and slave (like statement based on master and row based on slave) would cause inconsistencies in GTID generation.
To fix these and make the slave more robust, I introduced the following things:
  • While the slave is running a transaction, it will treat all tables as transactional tables when it comes to the caching of statements for the binary log.
  • Commits will only happen when the binary log says so.  This ensures that the slave will log and commit changes in the same order as the master, independent of the storage engine used.
  • CREATE is replayed on the slave as CREATE OR REPLACE.  This makes CREATE SELECT statements repeatable on the slave.
  • DROP TABLE statements are replayed on the slave as DROP TABLE IF EXISTS.  This makes DROP TABLE statements repeatable on the slave.
  • One can now have a mix of DDL and DML statments in the binary log (we use this fact to handle CREATE ... SELECT which is logged as BEGIN ; DROP; CREATE ; INSERTS ; COMMITS). This can be very useful also for other things in the future.
The end effect of the above is that the slave in 10.0.8 is going to be more robust than ever before.  In addition, the replication mode will not affect how GTID's are generated anymore.

I also added a variable 'slave-ddl-exec-mode' that one can set to STRICT if one prefers the old behavior that the the slave will fail if the DDL would fail on the slave for any reason, including if it fails to repeat a command on restart.

As a bonus, I also fixed that if one used LOCK TABLES with CREATE OR REPLACE TABLE, the lock will be held while the table is deleted and re-created and the lock is then added to the new table. This makes it possible to replace a table with an empty one without other users noticing it.

Here is an extract from the documentation of CREATE OR REPLACE :

The CREATE OR REPLACE TABLE syntax was added in MariaDB 10.0.8 to make replication more robust if it has to rollback and repeat statements like CREATE ... SELECT on slaves.

CREATE OR REPLACE TABLE table_name (a int);

is basically the same as:

DROP TABLE IF EXISTS table_name;
CREATE TABLE table_name (a int);


with the following exceptions:
  • If table_name was locked with LOCK TABLES it will continue to be locked after the statement.
  • Temporary tables are only dropped if the TEMPORARY key word was used. (With DROP TABLE temporary tables are preferred to be dropped before normal tables).