Comments on: Designing Databases: Picking The Right Data Types

By: Purencool

Purencool — Tue, 24 Nov 2009 01:33:06 +0000

Thanks for the great article. I have always wondered about the topic you discussed and what happens in large data sets

By: Brice Burgess

Brice Burgess — Tue, 24 Nov 2009 00:36:53 +0000

Fine introduction — although I side on an enum happy approach as it does a good job in intrinsically documenting the schema — but I would LOVE to hear more about approaches to handling DATES. Specifically dates pre 1970.

Keep up the good posts :)

By: Hodicska Gergely

Hodicska Gergely — Sat, 21 Nov 2009 23:36:45 +0000

“if you allow for usernames up to 12 characters, your VARCHAR should not be 255 characters wide.”
This is not true: varchar(12) and varchar(255) is the same as the length is stored on one byte in both cases. The change is here when you pass 255. It is always a good practice to use a bigger (under 255) value with varchar if you are not really sure that you wont need more to avoid unnecessary schema change.

“if you have 3 million records, could take you a week or two (hyperbole).”
This is very-very hyperbolist ;), it takes usually few minutes, of course this depends on the given setup.

“It’s poor schema design, because you’re expecting the database to do what PHP should have done for you.”
I disagree with you: you should defense you data model even on database level, while it is possible that not your code is the only one which interacts with the database. Of course you should consider other factors too like the cost of a possible schema modification, but for example with an MMM setup you can do this even with a running system without an outge.

“See slide 13 of the Drunken Query Master talk where he talks about having to rebuild the table”
I am a big fun of Jay Pipe, but you should believe the manual too. ;)

By: Herman Radtke

Herman Radtke — Sat, 21 Nov 2009 05:30:57 +0000

No, one of the most crucial, and most overlooked, components of database development, is the use of natural keys. The use of auto_increment and uuid/guid should be minimal.

The information on TEXT data types is just plain wrong. Look at http://dev.mysql.com/doc/refman/5.0/en/storage-requirements.html for reference. The only real difference between VARCHAR and TEXT is that VARCHAR has a length constraint.

Hard drives are cheap. RAID, proper indexes and caching are proper solutions to I/O bottlenecks.

For a discussion on what really matters in SQL design, I would suggest checking out Joe Celko’s book on SQL Style: http://www.amazon.com/exec/obidos/ASIN/0120887975

By: Jesper Wisborg Krogh

Jesper Wisborg Krogh — Sat, 21 Nov 2009 01:28:00 +0000

One thing also to be aware of is that how data types are handled varies between storage engines. InnoDB effective stores both CHAR and VARCHAR the same way. Also InnoDB at least in MySQL 5.0 uses four bytes to store a medium int (and I think small int as well) in which case I believe it is better to choose an int.

Regarding enum and sets, then I prefer not to use them and rather use a reference table (and for sets a join table) as it is my impression those data types are MySQL specific.

By: Dave Rowe

Dave Rowe — Fri, 20 Nov 2009 21:15:38 +0000

Interesting points. Though, in most cases, schema changes aren’t necessarily organic, thus a design may need to handle the ‘what ifs’ and potential growth issues. Deciding on SMALLINT versus BIGINT because of a few bytes isn’t really a concern, because the difference is only really seen in _large_ databases, at which point, this point is moot, since you’ll need BIGINTs to handle those millions of rows.

Design ideas themselves, should also be organic. As was demonstrated with your recommendation of avoiding ENUMs because of table rebuilds. Advancements in the technology allow for changes without the re-builds as referenced by FractalizeR, so the resulting point should be, make the best decision given what the underlying technology supports. But, be willing to be flexible on old rules.

Good post though! Keep ‘em coming.

By: Michael Crumm

Michael Crumm — Fri, 20 Nov 2009 20:51:49 +0000

Brandon,

Thanks for the post – I’ve often wondered the difference between certain MySQL data types, and this writeup did a nice job of explaining them.

By: Ryan

Ryan — Fri, 20 Nov 2009 16:19:52 +0000

Database-enforced constraints (like enum values) are, quite frankly, safer. When the database itself does the constraining, you (or another programmer) won’t inadvertently screw up data with bad code.

An acceptable compromise, I think, is to enforce enum values in a db layer. If the db layer knows it’s an ‘enum’ and what the accepted values are then no one can mess up the data*.

(Doctrine ORM defaults to simulated enums with varchar columns as not every DBMS supports enums).

* So then you run into possible issues when/if you do any raw SQL on your database (from CLI, scripts, whatever).

By: Jon

Jon — Fri, 20 Nov 2009 15:44:49 +0000

I think the bigger problem here is, Brandon, you’re trying to tell people how to design DB’s. No offense, but you aren’t a DBA, and DBA’s should be the ones to make those decisions. Not developers.

When developers design tables and databases, they design them to suit their needs for their application. This is very bad philosophy to follow. But if you must create your own tables because you have not a DBA, I would highly recommend that you learn Normalization rules, and get some books on MySQL best practices.

Some of this article clears up some confusion that intermediate developers will encounter. Some of the document makes assumptions based on “well..he said this so it must be true”. I think that unless you fully understand the issue and the causes, you shouldn’t be writing articles as a “matter of fact” when it’s a “matter of hearsay”.

By: Ben

Ben — Fri, 20 Nov 2009 12:30:52 +0000

I don’t claim to be any kind of MySQL or PHP authority, however, removing dependencies between database and application is a basic design principle – surely normalising the enum into a separate table is a better alternative to – presumably – using PHP constants to achieve the same result?

With regards to rebuilding the table – even if this is the case, it’s a single operation, and not something that will affect the overall performance of the database.

By: FractalizeR

FractalizeR — Fri, 20 Nov 2009 12:25:07 +0000

Dear Brandon, with all respect, I think blindly follow respected persons is a bad practice. Those slides you mention date back to 2008. And they are correct, in MySQL 5.0, there WAS a problem with ENUM and SET alerations leading to table restructures. Fast ALTER TABLE, as I understand, was implemented in 5.1 (in 5.0 manual there is no such entry, it appeared in 5.1 only).

By: Brandon Savage

Brandon Savage — Fri, 20 Nov 2009 12:16:06 +0000

Ben and FractalizeR, I’m sorry, but when it’s Jay Pipes (who works for MySQL) against you, I’m going to go with him every time. See slide 13 of the Drunken Query Master talk where he talks about having to rebuild the table.

It’s poor schema design, because you’re expecting the database to do what PHP should have done for you.

By: Ben

Ben — Fri, 20 Nov 2009 10:39:00 +0000

Personally, I don’t see the problem with using enum – if you have a case where a field can be one of two or three values it keeps your data readable without having to cross reference your code in order to understand exactly what is in a particular row.

If you must forbid the use of enums, the best alternative is to normalise the table, and extract the enum field into a seperate table – that way you can add an extra value easily – and it removes dependency between your database and your code.

Regardless, I dissagree with you on the inconvenience of adding an extra enum option at a later stage. It is a single operation, which, even if it takes several minutes, you can make the deployment at a time when the server isn’t busy – if you must deploy to a live database. Surely long term maintainability is more of a consideration than the length of time it will take to perform a single deployment?

By: FractalizeR

FractalizeR — Fri, 20 Nov 2009 09:36:09 +0000

>If you ever decide you have to add something else to the ENUM or SET declaration, MySQL must rebuild the entire table which, if you have 3 million records, could take you a week or two (hyperbole). This is clearly not optimal.

This is quite incorrect. If you add new values TO THE END of the ENUM or SET declaration, table rebuild is not needed: http://dev.mysql.com/doc/refman/5.1/en/alter-table.html (see fast table alterations).

By: Dominik

Dominik — Fri, 20 Nov 2009 08:57:51 +0000

One thing that I don’t understand – yet – is, why there are no “hexdata” or “uuid” fields. For example you mention md5 hashes and say that they are always CHAR(32). Sadly, that’s true, because it’s the only existing way to store them at this moment.

However actually md5 is a sequence of 16 “byte size” hex numbers, which could be stored in a 16 byte long “BSOB” (as in “Binary SMALL Object” :) – or “BIN” or “HEXDATA” or whatever you might want to call it). Same applies to UUIDs (which are also 16 bytes, just having some dashes in the hex representation).

Why is there no data type to put those in?