Now at LinuxFest NW listening to John Locke's talk about the Hacking business model
John Locke is owner of Freelock Computing, a small business in Seattle who is concentrating his business around Drupal. (Drupal is a tool that allows you to quickly develop web sites). John is following many of the same principles that are described in the Hacking business model that Zak Greant and I created based on our experience with the early days of MySQL Ab. Back then we followed many of the principles, including shared copyright, but the principles were never clearly written down and was over time abolished/ignored by managers who didn't understand them.
John did actually get so inspired by our document that he came and visited me in Finland in March and spent several days discussing business models, open source, and how to keep ones employees happy. We also enjoyed some good food, "some" black vodka, sauna and rolling around in the snow naked.
Ralf Wahlsten and I also spent some time going through John's business and helped him focus on the right things to go forward. He now seems to be on track and if you are in the Seattle area and need some good people to develop or help develop your web infrastructure I recommend you to contact his company.
Monty Program Ab is following the Hacking business model to the letter and it will be interesting to see how things will work out. I will keep posting about this to let you know what works and what doesn't work and the challenges we face as we grow.
An interesting thing is that we get many(!) work applications based on the Hacking business model alone! People mail us and say that Monty Program Ab is the type of company they always dreamt on working for. If you are about to start a company based on open source ideals, I recommend you subscribed to our model; It seams to be a sure way to attract good talent!
If you already have a company that has a similar model, or are already following the hacking business model, let us know about it and comment about your experiences!
I will update the the Hacking business model with links to companies who are following our principles!
2009-04-26
2009-04-21
To be (free) or not to be (free)
Tonight at 4:30 AM, USA Pacific time, my phone started to ring; it was a call from a Sun employee saying that Oracle has bought Sun and he wanted to join Monty Program Ab.
Shortly after that I got a call from a Swedish newspaper, Computer Sweden, who asked me about my opinion about why Oracle would buy Sun.
The reasons I see why Oracle is buying Sun are:
What could be Oracle's plan with MySQL? Three different plans come to mind:
I am putting my hopes to the third option, but for succeeding in that Oracle has to also learn a lot about open source development and working with the community.
This brings up the question, once again, how can one own an Open Source Project. Patrick Galbraith, summed up his feelings in a recent blog post, , that the "ownership" of Free and open source projects has more to do with who provides the best stewardship of the code, rather than who owns a trademark.
I don't think that anyone can own an open source project; the projects are defined by the de-facto project leaders and the developers that are working on the project. If the company loses the trust of these people, they can go away and fork the project and turn it the way they want to.
Sun's acquisition of MySQL did not go smoothly; most of the MySQL leaders (both commercial and project) have left Sun and the people who are left are sitting with their CV and ready to press send.
Oracle, not having the best possible reputation in the Open Source space, will have a hard time keeping the remaining MySQL people in the company or even working on the MySQL project. Oracle will also have a hard time to ensure to the MySQL customers, community and users that it will keep MySQL "free and available for all".
Here I see where Monty Program Ab, can play a significant role. Since I left Sun, I have been working on making it to be for Sun what Fedora is for RedHat. With Oracle now owning MySQL, I think that the need for an independent true Open Source entity for MySQL is even bigger than ever before.
The biggest threat to MySQL future is not Oracle per se, but that the MySQL talent at Sun will spread like the wind and go to a lot of different companies which will set the MySQL development and support back years.
I would not like to see this happen and I am doing everything I can do to keep this talent pool together (after all, most of them are long time personal friends of mine). I am prepared to hire or find a good home (either at Monty Program Ab or close to it) for all core MySQL personnel.
I am looking forward to working closely with Oracle (or whoever in the end gets to own MySQL) to ensure that there always exists a free branch of MYSQL that is actively develop in an open manner and has that trust and support of the MySQL customers, developers and users.
Mr. Ellison, you are undoubtably a master tactician. However, thinking two moves ahead in the open source world is not good enough. You need strategy. Long term, meaningful, viable strategy. You need to think years ahead, not just to the next fiscal quarter.
I'd love to speak with you about it.
Shortly after that I got a call from a Swedish newspaper, Computer Sweden, who asked me about my opinion about why Oracle would buy Sun.
The reasons I see why Oracle is buying Sun are:
- Sun is making big hardware, that is easy to bundle with very big Oracle installations.
- Sun is making very good, reliable Intel boxes that work well for database usage.
- Sun's virtualization product, VirtualBox
- Sun tape Storage, very well suited for efficiency database backups etc.
- Sun has done a lot of advanced work in cloud computing (even if Sun has not yet been able to monetize it)
- OpenSolaris, that would be a much better offer to Oracle's customers than their Oracle Linux platform, which doesn't provide any notable value over RedHat.
- Java
- And, of course, MySQL
What could be Oracle's plan with MySQL? Three different plans come to mind:
- They are going to kill MySQL (either directly or by not developing/supporting it fully)
- MySQL will get sold of to another entity, either because Oracle doesn't want it or becasue of anti-trust laws.
- They will embrace MySQL and Open Source and put their technical expertise on it to ensure that MySQL continues to be the most popular advanced Open Source database.
I am putting my hopes to the third option, but for succeeding in that Oracle has to also learn a lot about open source development and working with the community.
This brings up the question, once again, how can one own an Open Source Project. Patrick Galbraith, summed up his feelings in a recent blog post, , that the "ownership" of Free and open source projects has more to do with who provides the best stewardship of the code, rather than who owns a trademark.
I don't think that anyone can own an open source project; the projects are defined by the de-facto project leaders and the developers that are working on the project. If the company loses the trust of these people, they can go away and fork the project and turn it the way they want to.
Sun's acquisition of MySQL did not go smoothly; most of the MySQL leaders (both commercial and project) have left Sun and the people who are left are sitting with their CV and ready to press send.
Oracle, not having the best possible reputation in the Open Source space, will have a hard time keeping the remaining MySQL people in the company or even working on the MySQL project. Oracle will also have a hard time to ensure to the MySQL customers, community and users that it will keep MySQL "free and available for all".
Here I see where Monty Program Ab, can play a significant role. Since I left Sun, I have been working on making it to be for Sun what Fedora is for RedHat. With Oracle now owning MySQL, I think that the need for an independent true Open Source entity for MySQL is even bigger than ever before.
The biggest threat to MySQL future is not Oracle per se, but that the MySQL talent at Sun will spread like the wind and go to a lot of different companies which will set the MySQL development and support back years.
I would not like to see this happen and I am doing everything I can do to keep this talent pool together (after all, most of them are long time personal friends of mine). I am prepared to hire or find a good home (either at Monty Program Ab or close to it) for all core MySQL personnel.
I am looking forward to working closely with Oracle (or whoever in the end gets to own MySQL) to ensure that there always exists a free branch of MYSQL that is actively develop in an open manner and has that trust and support of the MySQL customers, developers and users.
Mr. Ellison, you are undoubtably a master tactician. However, thinking two moves ahead in the open source world is not good enough. You need strategy. Long term, meaningful, viable strategy. You need to think years ahead, not just to the next fiscal quarter.
I'd love to speak with you about it.
2009-03-24
Thanks for all the meat
I had last Friday an extremely nice 'farewell to Sun' (for me as an employee) dinner with former and current Finnish MySQL/Sun employees.
For once we didn't have the dinner at my place, because for this particular occasion I didn't want to spend a major part of my time in the kitchen (which I usually do because I love to cook).
We were at a Russian restaurant 'Saslik', which is famous for it's great food, hospitality and cold vodka (they took one look at us and put a full bottle of more-than-ice-cold Russian vodka on the table).
We had an entree called 'Zakuska' (a combination of Russian appetizers). Most of us took "Ivan's sword" as the main course and as desert we had "Baked Alaska à la Catherine the Great".
If you live in or are visiting Finland and want to have a special and memorable dinner, the above is what I would recommend (assuming you like meat).
Thanks to all that showed up at dinner (almost everyone was able to attend)!
Thanks Sun for the dinner!
From the development side, I am happy to announce than we are just adding a feature into MariaDB that will make MariaDB to perform much better on Solaris (thanks to the getport system call).
A special thanks to Henrik Ingo who had on his blog started a collection to 'the personal MySQL memory' where a lot of people have shared their memories about the old times and their encounters with MySQL and me.
Reading the blog was a great walk down memory lane. Thanks a lot for everyone that has contributed to it and everyone else who will contribute to it in the future!
What I like most about open source/free software and open development is that you don't lose contact with your former colleagues, friends or contacts just because you change work! Being able to continue to work on the same project and with the same people regardless of where you are is a great strength that is unique to open source.
What I find most delightful with now being outside of Sun is that I have been able to reconnect with a lot of former MySQL employees; Arjen Lenz, Peter Zaitsev, Kristian Nielsen, Ronald Bradford, Tonu Samuel, Antti Halonen, Larry Stefonic, Indrek Siitan, Patrik Backman, Camilla Zilliacus just to name a few (in random order) who I have talked with during the last weeks.
MySQL AB has always kept former employees and even companies working on anything that 'may compete' with MySQL business at arm length, which I have always thought 'leaves a lot to be desired'. By doing that, we have constrained the MySQL business growth instead of cooperating and making our total market larger. Many companies put pride in having an alumni club, but not yet the former MySQL AB.
What I think is needed to take MySQL to the next level is to create a company that is dedicated to support the community of the MySQL users, the MySQL developers (regardless of where they work) and all the companies that develop, work on or provide services for or upon MySQL. It is difficult for Sun to do this as the commercial interest of the different companies hinders trust between the parties and Sun.
This is what I am trying to do with Monty Program Ab.
I think we are uniquely suitable for this because:
- The people at Monty Program Ab have a great track record in working with the community.
- We are clearly dedicated to open source.
- We have proven over and over that our ideals can't be bought. We are working for open source and open source ideals both in good and bad times!
- We have a clear business agenda which is not conflicting with any other MySQL company.
- We are focusing on doing paid MySQL development, the result of which we will release in our open source MariaDB release.
- We do binary releases early and often, and we don't differentiate versions between customers and community users.
- We are creating a network of MySQL companies to together serve ALL needs of MySQL users.
- We acknowledge Sun as the key owner of the intellectual property rights.
- We are not doing front line support, training, web development etc.
A lot of companies have already contacted me to be partners in the network. Some of them can be found here.
If you want to be part of this new network around MariaDB / MySQL, send us an email to 'partners (at) a s k m o n t y (dot) o r g'.
If you need help with anything related to MySQL or databases, feel free to contact us at 'i n f o (at) a s k m o n t y (dot) o r g'. If we can't provide the right solution for your problem, we should be able to find a partner or someone in the MySQL community to help you!
And of course, if you are a developer who wants to change MySQL to better satisfy your business needs, join us developing MariaDB.
Lets connect to create better MySQL experiences for everyone!
For once we didn't have the dinner at my place, because for this particular occasion I didn't want to spend a major part of my time in the kitchen (which I usually do because I love to cook).
We were at a Russian restaurant 'Saslik', which is famous for it's great food, hospitality and cold vodka (they took one look at us and put a full bottle of more-than-ice-cold Russian vodka on the table).
We had an entree called 'Zakuska' (a combination of Russian appetizers). Most of us took "Ivan's sword" as the main course and as desert we had "Baked Alaska à la Catherine the Great".
If you live in or are visiting Finland and want to have a special and memorable dinner, the above is what I would recommend (assuming you like meat).
Thanks to all that showed up at dinner (almost everyone was able to attend)!
Thanks Sun for the dinner!
From the development side, I am happy to announce than we are just adding a feature into MariaDB that will make MariaDB to perform much better on Solaris (thanks to the getport system call).
A special thanks to Henrik Ingo who had on his blog started a collection to 'the personal MySQL memory' where a lot of people have shared their memories about the old times and their encounters with MySQL and me.
Reading the blog was a great walk down memory lane. Thanks a lot for everyone that has contributed to it and everyone else who will contribute to it in the future!
What I like most about open source/free software and open development is that you don't lose contact with your former colleagues, friends or contacts just because you change work! Being able to continue to work on the same project and with the same people regardless of where you are is a great strength that is unique to open source.
What I find most delightful with now being outside of Sun is that I have been able to reconnect with a lot of former MySQL employees; Arjen Lenz, Peter Zaitsev, Kristian Nielsen, Ronald Bradford, Tonu Samuel, Antti Halonen, Larry Stefonic, Indrek Siitan, Patrik Backman, Camilla Zilliacus just to name a few (in random order) who I have talked with during the last weeks.
MySQL AB has always kept former employees and even companies working on anything that 'may compete' with MySQL business at arm length, which I have always thought 'leaves a lot to be desired'. By doing that, we have constrained the MySQL business growth instead of cooperating and making our total market larger. Many companies put pride in having an alumni club, but not yet the former MySQL AB.
What I think is needed to take MySQL to the next level is to create a company that is dedicated to support the community of the MySQL users, the MySQL developers (regardless of where they work) and all the companies that develop, work on or provide services for or upon MySQL. It is difficult for Sun to do this as the commercial interest of the different companies hinders trust between the parties and Sun.
This is what I am trying to do with Monty Program Ab.
I think we are uniquely suitable for this because:
- The people at Monty Program Ab have a great track record in working with the community.
- We are clearly dedicated to open source.
- We have proven over and over that our ideals can't be bought. We are working for open source and open source ideals both in good and bad times!
- We have a clear business agenda which is not conflicting with any other MySQL company.
- We are focusing on doing paid MySQL development, the result of which we will release in our open source MariaDB release.
- We do binary releases early and often, and we don't differentiate versions between customers and community users.
- We are creating a network of MySQL companies to together serve ALL needs of MySQL users.
- We acknowledge Sun as the key owner of the intellectual property rights.
- We are not doing front line support, training, web development etc.
A lot of companies have already contacted me to be partners in the network. Some of them can be found here.
If you want to be part of this new network around MariaDB / MySQL, send us an email to 'partners (at) a s k m o n t y (dot) o r g'.
If you need help with anything related to MySQL or databases, feel free to contact us at 'i n f o (at) a s k m o n t y (dot) o r g'. If we can't provide the right solution for your problem, we should be able to find a partner or someone in the MySQL community to help you!
And of course, if you are a developer who wants to change MySQL to better satisfy your business needs, join us developing MariaDB.
Lets connect to create better MySQL experiences for everyone!
2009-02-24
MoiPal, making it more fun to learn things
It's really a hard job to decide into which companies to invest. We at Open Ocean, my investment company, have talked with around 60 companies the last few months to find those that we think have a great potential and that would benefit from our involvement.
One company that more than well matched our investment criterias was Ironstar, that makes the MoiPal virtual world. With more than 100,000 registered users they are definitely a company to watch out for.
I have always been interested in virtual worlds and see how the communities around them continue to grow. The thing that really stands out to me with MoiPal, is the way it combines the social networking aspects similar to Facebook, with the caring of your online character, like you do with Tamagotchis or in games like Sims.
MoiPal has also open interfaces and engages other developers to expand the MoiPal world in their own, often unexpected, directions. Open interfaces are extremely important for a software to succeed today; For example, it was thanks to its open interfaces that MySQL got connections to all major programming languages and tools! This was something we could never have done by ourselves.
I think it's important that you learn when you play (as long as it doesn't take away the enjoyment of the game!). I think it will be easy to add more educational aspects into MoiPal; For example by making it easier to communicate with other "Pals" that has other languages and backgrounds. By building bridges to make the world a 'smaller place' we make it easier for people to understand and eventually respect each other. I personally like to invest in ideas that have a good social aspect!
On the business side, I found it really funny, when I heard that Lordi performed a virtual concert in MoiPal for thousands of players wearing virtual Lordi masks. I can envision this to be a major channel to launch youth brands in the future. When we reach a million users, I would hope we could launch for instance the H&M Spring collection simultaneously in MoiPal and in the real world. Then youngsters could buy the same trendy outfit, both for themselves and for their Pal.
I know that the audience of this blog is unlikely to be users of MoiPal. I hope you will still find this information about MoiPal interesting; If you have young children it's likely that you will find them at MoiPal and then it's good to know that it's a good and safe place for them to be in!
One company that more than well matched our investment criterias was Ironstar, that makes the MoiPal virtual world. With more than 100,000 registered users they are definitely a company to watch out for.
I have always been interested in virtual worlds and see how the communities around them continue to grow. The thing that really stands out to me with MoiPal, is the way it combines the social networking aspects similar to Facebook, with the caring of your online character, like you do with Tamagotchis or in games like Sims.
MoiPal has also open interfaces and engages other developers to expand the MoiPal world in their own, often unexpected, directions. Open interfaces are extremely important for a software to succeed today; For example, it was thanks to its open interfaces that MySQL got connections to all major programming languages and tools! This was something we could never have done by ourselves.
I think it's important that you learn when you play (as long as it doesn't take away the enjoyment of the game!). I think it will be easy to add more educational aspects into MoiPal; For example by making it easier to communicate with other "Pals" that has other languages and backgrounds. By building bridges to make the world a 'smaller place' we make it easier for people to understand and eventually respect each other. I personally like to invest in ideas that have a good social aspect!
On the business side, I found it really funny, when I heard that Lordi performed a virtual concert in MoiPal for thousands of players wearing virtual Lordi masks. I can envision this to be a major channel to launch youth brands in the future. When we reach a million users, I would hope we could launch for instance the H&M Spring collection simultaneously in MoiPal and in the real world. Then youngsters could buy the same trendy outfit, both for themselves and for their Pal.
I know that the audience of this blog is unlikely to be users of MoiPal. I hope you will still find this information about MoiPal interesting; If you have young children it's likely that you will find them at MoiPal and then it's good to know that it's a good and safe place for them to be in!
2009-02-16
Life goes on and making the internet more secure with Web of Trust (WOT)
It's now more than a week since I left Sun and I have been very busy with old commitments; I had one talk at the Tampere University about "Open source licensing and how this affects quality" and a keynote about "Open Source Licensing" for the "2nd Symposium of the HyperTransport Center of Excellence" in Mannheim, Germany.
My web site, askmonty.org, is coming around nicely but it will take a couple of more weeks to add some missing information to it. After that I will start working on the Maria, MariaDB and MySQL code bases for real.
The most exiting thing that has happened so far, is that my investment company "Open Ocean" have just closed a funding round with "Web of Trust" i.e. WOT. I will take a seat on the company's Board of Directors.
What I like about WOT is that it solves a practical problem I have often experienced myself when I browse and want to buy things from Web sites: "Can I really trust this web site with my personal information, like my credit card". In addition WOT solves the problem in an elegant and user friendly way; Easy enough so that anyone can use it.
So what is then WOT? WOT is a popular and free browser add-on that works with Firefox and Explorer. It offers you information whether a web site is known to be involved in Internet scams, identity theft, spyware, spam or if it's just an unreliable online shop.
WOT provides safety ratings to search results when using Google, Yahoo!, Wikipedia, Digg and other popular sites. The Website rating information is continuously updated by the WOT user community and numerous trusted sources, such as listings of from numerous malware and phishing sites.
I encourage everyone to try out WOT to get a better Internet experience. You should consider registering as a WOT user to be able to rate web sites.
In addition, if you encounter a web site that is un-trustworthy, please rate it trough WOT to tell other Internet users about your experience to save them from the possibly trouble you suffered. Also, if you really like a web site or if you get excellent service from it, please use WOT and tell others about this!
I hope we can all work together and make the Internet a place where you can easily know where it's safe to browse and shop!
You can find a lot more information about WOT, including downloading it, from http://www.mywot.com. If you are using Firefox, you can install it by using the menu option "tools/add-ons", click on "get add-ons" and do a search after "WOT".
My web site, askmonty.org, is coming around nicely but it will take a couple of more weeks to add some missing information to it. After that I will start working on the Maria, MariaDB and MySQL code bases for real.
The most exiting thing that has happened so far, is that my investment company "Open Ocean" have just closed a funding round with "Web of Trust" i.e. WOT. I will take a seat on the company's Board of Directors.
What I like about WOT is that it solves a practical problem I have often experienced myself when I browse and want to buy things from Web sites: "Can I really trust this web site with my personal information, like my credit card". In addition WOT solves the problem in an elegant and user friendly way; Easy enough so that anyone can use it.
So what is then WOT? WOT is a popular and free browser add-on that works with Firefox and Explorer. It offers you information whether a web site is known to be involved in Internet scams, identity theft, spyware, spam or if it's just an unreliable online shop.
WOT provides safety ratings to search results when using Google, Yahoo!, Wikipedia, Digg and other popular sites. The Website rating information is continuously updated by the WOT user community and numerous trusted sources, such as listings of from numerous malware and phishing sites.
I encourage everyone to try out WOT to get a better Internet experience. You should consider registering as a WOT user to be able to rate web sites.
In addition, if you encounter a web site that is un-trustworthy, please rate it trough WOT to tell other Internet users about your experience to save them from the possibly trouble you suffered. Also, if you really like a web site or if you get excellent service from it, please use WOT and tell others about this!
I hope we can all work together and make the Internet a place where you can easily know where it's safe to browse and shop!
You can find a lot more information about WOT, including downloading it, from http://www.mywot.com. If you are using Firefox, you can install it by using the menu option "tools/add-ons", click on "get add-ons" and do a search after "WOT".
2009-02-05
Time to move on
I have now departed from Sun and joined my own company, Monty Program Ab.
There were a lot of rumors around me resigning in August/September last year. I didn't back then want to comment on the rumors, because I was still trying to work something out with Sun. Now I can finally describe a bit of what was going on.
In this case, the rumors had some elements of truth to them. I had told management that I thus would be submitting my resignation immediately as I strongly believed that the 5.1 release was not ready and that those problems needed to be fixed before it went GA. This action, together with other peoples´ efforts, did have the wanted effect and I made an agreement with Sun´s upper management to not initiate my resignation but instead stay around for three more months to help Sun work out things in MySQL Development and also give Sun a chance to create an optimal role for me within Sun.
The three months did stretch out to seven months, and the changes I had hoped Sun would apply to in the MySQL Database group to fix our development and community problems did not happen fast enough.
Sun and I concluded in the end that I have much higher chances of achieving my goals outside of Sun, so it's just better to swallow the bitter apple, go out and get things going. We parted in good terms and we both expect to continue to do business and work together.
The main reason for leaving was that I am not satisfied with the way the MySQL server has been developed, as can be seen on my previous blog post. In particular I would have like to see the server development to be moved to a true open development environment that would encourage outside participation and without any need of differentiation on the source code. Sun has been considering opening up the server development, but the pace has been too slow.
I still think that Sun was the best possible buyer for MySQL and I feel sad that things didn't work out together. Sun has a lot of good things going on and I hope that they will continue their path to create and promote Open source. I will be available for Sun in helping them with their goals in the Open source space.
So what will Monty Program Ab and I be working on?
Monty Program Ab will be a true open source company, with the additional goal of being a smaller family oriented company (10-30 employees) where everyone can be owners of the company, where we care about our employees and strive to have fun together and share the profit we create. You can find more about this at: http://zak.greant.com/hacking-business-models
Some of the Maria team members will follow me and some will stay in Sun. The plan is to continue to work on the Maria project more or less as before. The main difference from before is that the Maria project and it's mailing lists will move to launchpad and we will start using free-node for our IRC communications (channel #maria). Maria 1.5 (the crash safe version of MyISAM) is now in beta and we hope to get binaries out soon. We have already started working on Maria 2.0 features (full transactional release) and performance issues. From my side there will be a small time delay in the Maria development (one-two months delay in the final release) as I have to set up things in the new company.
Monty Program Ab will start working actively with the MySQL community to allow stable patches to the MySQL-5.1-Maria tree and create an active community around this tree. We will also do some restructuring of the MySQL code to make it simpler, faster and with fewer bugs.
One shouldn't regard the MySQL-Maria tree as a fork but as a branch as we intend to pull in all changes from the official tree to the MySQL-Maria tree; Some of the changes will probably be reworked but we will do our best to ensure that for the end user they should look and feel the same.
Monty Program Ab will also do NRE (Non-recurring engineering) to customers on MySQL and Maria and put this work into the MySQL-Maria tree. (Shameless promotion: If you want something to be done, fixed or changed in the MySQL server, please contact me at "monty at askmonty.org" for an offer).
In addition, I will continue to work with and invest in disruptive technology start Up companies that do open source and community products. (Another shameless promotion : If you think you have an unique business idea in the open source space, contact me on "monty at openoceancapital.com"!)
I am also working on opening a new kind of restaurant that will use databases to achieve a better customer experience. (Expect a special discount for database developers!). More about this later...
More information of what I will be doing will be found at http://www.askmonty.org or
http://www.openoceancapital.com
(Note that the above web sites will be under constructions for the next few weeks)
There were a lot of rumors around me resigning in August/September last year. I didn't back then want to comment on the rumors, because I was still trying to work something out with Sun. Now I can finally describe a bit of what was going on.
In this case, the rumors had some elements of truth to them. I had told management that I thus would be submitting my resignation immediately as I strongly believed that the 5.1 release was not ready and that those problems needed to be fixed before it went GA. This action, together with other peoples´ efforts, did have the wanted effect and I made an agreement with Sun´s upper management to not initiate my resignation but instead stay around for three more months to help Sun work out things in MySQL Development and also give Sun a chance to create an optimal role for me within Sun.
The three months did stretch out to seven months, and the changes I had hoped Sun would apply to in the MySQL Database group to fix our development and community problems did not happen fast enough.
Sun and I concluded in the end that I have much higher chances of achieving my goals outside of Sun, so it's just better to swallow the bitter apple, go out and get things going. We parted in good terms and we both expect to continue to do business and work together.
The main reason for leaving was that I am not satisfied with the way the MySQL server has been developed, as can be seen on my previous blog post. In particular I would have like to see the server development to be moved to a true open development environment that would encourage outside participation and without any need of differentiation on the source code. Sun has been considering opening up the server development, but the pace has been too slow.
I still think that Sun was the best possible buyer for MySQL and I feel sad that things didn't work out together. Sun has a lot of good things going on and I hope that they will continue their path to create and promote Open source. I will be available for Sun in helping them with their goals in the Open source space.
So what will Monty Program Ab and I be working on?
Monty Program Ab will be a true open source company, with the additional goal of being a smaller family oriented company (10-30 employees) where everyone can be owners of the company, where we care about our employees and strive to have fun together and share the profit we create. You can find more about this at: http://zak.greant.com/hacking-business-models
Some of the Maria team members will follow me and some will stay in Sun. The plan is to continue to work on the Maria project more or less as before. The main difference from before is that the Maria project and it's mailing lists will move to launchpad and we will start using free-node for our IRC communications (channel #maria). Maria 1.5 (the crash safe version of MyISAM) is now in beta and we hope to get binaries out soon. We have already started working on Maria 2.0 features (full transactional release) and performance issues. From my side there will be a small time delay in the Maria development (one-two months delay in the final release) as I have to set up things in the new company.
Monty Program Ab will start working actively with the MySQL community to allow stable patches to the MySQL-5.1-Maria tree and create an active community around this tree. We will also do some restructuring of the MySQL code to make it simpler, faster and with fewer bugs.
One shouldn't regard the MySQL-Maria tree as a fork but as a branch as we intend to pull in all changes from the official tree to the MySQL-Maria tree; Some of the changes will probably be reworked but we will do our best to ensure that for the end user they should look and feel the same.
Monty Program Ab will also do NRE (Non-recurring engineering) to customers on MySQL and Maria and put this work into the MySQL-Maria tree. (Shameless promotion: If you want something to be done, fixed or changed in the MySQL server, please contact me at "monty at askmonty.org" for an offer).
In addition, I will continue to work with and invest in disruptive technology start Up companies that do open source and community products. (Another shameless promotion : If you think you have an unique business idea in the open source space, contact me on "monty at openoceancapital.com"!)
I am also working on opening a new kind of restaurant that will use databases to achieve a better customer experience. (Expect a special discount for database developers!). More about this later...
More information of what I will be doing will be found at http://www.askmonty.org or
http://www.openoceancapital.com
(Note that the above web sites will be under constructions for the next few weeks)
2008-11-29
Oops, we did it again (MySQL 5.1 released as GA with crashing bugs)
MySQL 5.1 is now released as "GA".In this blog I will try to describe my opinions about this release and also try to set the expectations right for anyone trying out MySQL 5.1 GA.
What should you then expect from MySQL 5.1?
- If you are using MySQL 5.1 just as a 'better' version of MySQL 5.0 and you don't plan to use any of the new features in MySQL 5.1 then you are probably fine to try out MySQL 5.1. You should however not put it into production without testing it fully, preferably by running it on a couple of slaves for some weeks. It may even be the best to wait for a couple of minor/patch releases before putting the MySQL 5.1 server into production.
- Don't expect that all critical bugs that you may have encountered in 5.0 to be fixed in 5.1. Even if we have fixed a big majority of the bugs from 5.0 some really critical ones still haven't been addressed.
- If you plan to use any of the new features of MySQL 5.1, regard these as if they would be of beta quality. Test any usage of these features extensively for in close-to-live scenarios before putting them onto a production server.
- If you are a new user trying out MySQL for the first time, you should use MySQL 5.1; At least it's better than the MySQL 5.0 community version which has not been updated for some time.
To prove my points, here is some metrics and critical bugs for 5.1:
- We still have 20 known and tagged crashing and wrong result bugs in 5.1 35 more if we add the known crashing bugs from 5.0 that are likely to also be present in 5.1.
- We still have more than 180 serious bugs (P2) in 5.1. Some of these can be found here.
- We have more than 300 known and verified less critical bugs that are not going to be addressed soon. (The total reported number of bugs to the MySQL server is of course much larger)
- Bug #989 "If DROP TABLE while there's an active transaction, wrong binlog order". This is a bug that has been known since August 2003, and has been discussed an referred to in several public places, including Wikipedia and my last talk at the MySQL users conference. It allows in effect anyone with rights to any database that is replicated to take down all slaves (either by accident or intentionally). This is also a bug that has been hit by several of our users in the past.
- Bug #33082 Stored Procedure: crash if table replaced with a view in a loop
- Bug #33094 Error in upgrading from 5.0 to 5.1 when table contains triggers
- Bug #34110 Crash in InnoDB when used "embedded"
- Bug #34502 mysqladmin debug causes a crash when server is creating/dropping many tmp tables
- Bug #34660 crash when federated table loses connection during insert ... select
- Bug #37756 enabling fulltext indexes with MyISAM_repair_threads > 1 causes crash
- Bug #37936 "Crash when executing a query containing date expressions"
- Bug #38816 kill + flush tables with read lock + stored procedures causes crashes!
- Bug #39178 Server crash in YaSSL with non-RSA-requesting client if server uses RSA key
- Bug #40386 Not flushing query cache after truncate
- Bug #40675 MySQL 5.1 crash with index merge algorithm and Merge tables
- Bug #32868 Stored routines do not detect changes in meta-data. Note that this will not be fixed until 6.1 !
- Bug #39526 sql_mode not retained in binary log for CREATE PROCEDURE
- The federated engine is not enabled by default. It was disabled during a previous MySQL-5.1 "RC" release because of bugs filed against the Federated engine that MySQL developers didn't have time to fix. This solution was deemed to be easier than upgrading the Federated engine to a newer version of the engine. This means that people that have problems with the federated engine are better off using a FederatedX plugin, compiling MySQL them self together with FederatedX or use the ourdelta MySQL distribution which contains FederatedX.
- MySQL-Cluster bugs are not fixed in MySQL 5.1; Instead the Cluster engine is moved from the MySQL 5.1 release to a separate MySQL-Cluster release.
Partitioning:
- 20 open bugs of which at least 7 are targeted to be fixed in later MySQL 5.1 releases.
- Partitioning in MySQL 5.1 should be regarded as a step to a full partitioning feature with parallel query. Parallel query is however not scheduled even for MySQL 6.0.
- For now partitioning is mainly useful in the case where you need to frequently drop a well defined part of a table (like one month of data) and when MERGE tables are too cumbersome to use.
- If one partitioned table crashes it's very hard (sometimes impossible to repair it.
- If you get a server crash during ALTER TABLE of a partitioned table you may loose all your data for that table.
- Partitioning is very slow and can become unusable if you have a large number of partitions. This happens even if you only use a few of the underlying tables in your query.
- Bug #40954 "Crash in MyISAM index code with concurrency test using partitioned tables"
- Bug #40827 Killing insert-select from InnoDB to partitioned MyISAM can cause table corruption
- Bug #30102 rename table does corrupt tables with partition files on failure
Row based replication has been regarded as one of the most wanted feature in 5.1. However, because of several problems with the implementation of row based and mixed mode replication it's not enabled by default. These problems are:
- At least 28 open bugs of which 26 are verified and at least 11 are targeted to be fixed in later MySQL 5.1 releases.
- Row based replication errors on the slave can be hard to debug as you can't see exactly what statement caused the problem. A new feature in MySQL 5.1.28 allows you to see what rows was changed, but this is usually not enough to find out the exact query that failed to replicate.
- For bulk operations on non transactional tables, the data may appear inconsistent during selects on the slave (source MySQL manual)
- Bug #40221 Replication failure on RBR + UPDATE the primary key. This bug is such a serious issue that it should have stopped a GA release!
- Bug #38205 Row-based Replication (RBR) causes inconsistencies: HA_ERR_FOUND_DUPP_KEY. This causes wrong data on slave if you do slave start/stop at the wrong time.
- Bug #40116 Uncommitted changes are replicated and stay on slave after rollback on master
- Bug #40276 Assertion trx_data->empty() in binlog_close_connection
- Bug #31240 load data infile replication between (4.0 or 4.1) and 5.1 fails
It's hard to find a number of bugs on events as there is no easy way to search for them in the bugs system. In general events are regarded to be one of the more stable features in MySQL, but it's not totally free from problems:
- Bug #40915 "Events takes mutex in wrong order which can easily lead to deadlocks"
This was part of the announcement but I don't know that we mean with this. I couldn't find anything about this on the "What's New in MySQL 5.1" page. I assume this refers to the SHOW PROFILE patch from MySQL 5.0 community which is now in MySQL 5.1
Logging to tables:
This major feature is unfortunately so slow (30% + slowdown) that it's unusable for busy sites. Ref: Bug #30414: "Slowdown (related to logging) in 5.1.21 vs. 5.1.20". I assume this was why it was left out from the MySQL announcement of MySQL 5.1
Some general crashing/wrong data bugs (not all, just enough to prove a point):
- Bug #40770 Server Crash when running with triggers including variable settings (rpl_sys)
- Bug #37016 TRUNCATE TABLE removes some rows but not all
So what went wrong with MySQL 5.1 ?
This is surprisingly not because our developers don't do a good job. On the contrary we have an excellent dedicated team of developers that are very good in what they are doing. However, even an excellent team can't work if the conditions are not right.
Here follows some of the main reasons why MySQL development department again
got a quality problem with a GA release:
- MySQL 5.1 was declared beta and RC way too early. The reason MySQL 5.1 was declared RC was not because we thought it was close to being GA, but because the MySQL manager in charge *wanted to get more people testing MySQL 5.1*. This didn't however help much, which is proved by the fact that it has taken us 14 months and 7 RC's before we could do the current "GA". This caused problems for developers as MySQL developers have not been able to do any larger changes in the source code since February 2006!
- We have changed the release model so that instead of focusing on quality and features our release is now defined by timeliness and features. Quality is not regarded to be that important. To quote Mårten Mickos: "MySQL 5.1 will be release as GA in or before December because I say so". Mårten's reasons for this is that he needs something he can sell and a release marked "GA" is much easier to sell than a release marked "RC".
- The MySQL core developers have been split into too many teams and only a small part of the core developers have been working on MySQL 5.1 to get the bugs fixed. Some of the core developers have also recently left the MySQL organization which is a serious issue as there is not many of of them.
- Too many new developers without a thorough knowledge of the server have been put on the product trying to fix bugs. This in combined with a failing review process have introduced of a lot new bugs while trying to fix old bugs.
- Bug fixing and development processes are not systematic and not persistent.
- We have not been giving the MySQL community enough opportunities to test MySQL 5.1 (too few releases). The reason few releases was made was that if we would have done a release every month, as we have done in the past, we would have got 14 RC releases which would have looked silly and proved that the first RC was made too early. In addition, the MySQL current development model doesn't in practice allow the MySQL community to participate in the development of the MySQL server.
- The MySQL organization doesn't have a release criteria for the MySQL server that is followed; Both the external one and the internal one have not been followed when it comes to declaring MySQL 5.1 as GA. You can read more about our release policy in Kaj's blog.
- Internal QA on the MySQL server was started very late in the process. Now when the process have started to show results, the found bugs have largely being ignored as fixing these they would delayed the MySQL 5.1 GA date.
- The MySQL server team have a bug fixing policy where a bug that has existed a long time has a lower priority 'because people know about them'. This is supposedly one of the reasons why the Bug#989 mentioned above has not been fixed.
What should then have been done before declaring MySQL GA ?
It's of course impossible to get all issues fixed, but we should at least have tried to ensure that all issues important to a lot of MySQL developers and MySQL users should have been discussed, fixed and/or addressed in a public manner! We should also never have a single serious crashing/wrong data bug in a GA release.
There should also be, from MySQL management, an independent release criteria committee that would be the one deciding when the MySQL server is ready to be declared beta, RC and GA. This is something that Sun usually has for their other products.
As I said in my talk at the MySQL users conference, I think it's time to seriously review how the MySQL server is being developed and change the development model to be more like Drizzle and PostgreSQL where the community has a driving role in what gets done!
I would like to point out that the current release is not something that can be said to be fault of Sun. The decisions to do a GA release was solely been made by the MySQL management in Sun. The only thing Sun can be blamed of is to not start fixing the MySQL development organization soon enough to ensure that things like this can't happen.
I still have some hopes that Sun will come in and fix the MySQL development organization, but with MySQL server releases like this one my hopes have started to fade a bit.
There is however some good news in getting MySQL 5.1 released as GA:
- MySQL community users that have not got an update for MySQL 5.0 for 4 months should be able to switch to MySQL 5.1 and now finally get some of their bugs fixed! What still worries me is that the MySQL organization has not yet clearly defined how future MySQL 5.1 versions will be released to the community. This is however a large topic of it's own...
- The MySQL embedded library is back in a supported release. (Not a big thing, but still important for some part of our community).
Install and test MySQL 5.1; If it works, feel lucky. If not, report a bug at http://bugs.mysql.com/. Don't forget to blog about your experiences with MySQL 5.1!
There is two ways things can go:
- If MySQL 5.1 works for a lot of people and not too many get serious crashes and losses data, then I was concerned without a good reason and everything is fine.
- If MySQL 5.1 does have some serious problems and people report them, the bugs will be fixed and the MySQL & Sun management will have more information to not repeat the same thing with MySQL 6.0.
PS: For those interested, the picture in the blog is decidicated to those in charge of releasing MySQL 5.1 as GA. The statue itself was bought in Riga this year during the internal MySQL developers conference and was presented to the MySQL 6.0 managers as a symbol for the MySQL 6.0 server release planning.
Subscribe to:
Posts (Atom)
Posts
