Preventing Drupal Headaches! The file field
In our structured Drupal training program, we teach the "right way" to do things. We listen to our support team of client advisors who see all the permutations of "you're doing it wrong" in Drupal, and make sure we're leading learners in the right direction. When there's something particularly essential we use words like "Important" and set off warnings with a little exclamation mark icon. What we don't do is surround particular instruction in skulls and cross-bones and "Ye shall not pass" signs explaining the hells that will befall you if you don't do something a certain way. Because everything in Drupal is optional, and up to you.
Take, for example, the file field "File directory" path settings. If you didn't know it could lead you to potential disaster, you might think this "optional" setting is something you could just overlook. Many clients add file fields, and never complete this configuration. With multiple content types, files on taxonomy terms or other entities, it means ALL the uploaded files across the site are going into the same directory.
In fact, we encounter this problem while supporting some of the biggest and busiest websites with our hosting business. "How many files can you store in a directory" depends on your server software and configuration. Like the proverbial "clowns in a car" trick, you don't want to attempt to reach that upper limit. While the technical upper limits may be high, you can notice performance implications with 1000 or 100,000 files. This could be when you try to load a directory, or even display an image from a directory. You may never think you'll get up to that many files, but if you have users generating content, you can get surprised.
Especially if you haven't let Drupal handle the file referencing and management, remediating the problem is painfully slow. For example, adding inline references to images on your server, instead of using modules like Media which use tokens to place images inline. My colleague, Devin Carlson, (who is very knowledgeable about the file system in Drupal 7) explained it took him 1-2 weeks to fix a similar problem on a client site. He used the Migrate module, wrote custom scripts and tested over and over again to ensure all the files were moved and all internal inline references were updated.
How to add file upload fields to Drupal the right way
In terms of file path management, you can do three things:
- Even if you don't install another module you can add a prefix to each instance of a file field. For example, add the entity name (content type, taxonomy term name).
- Use the Token module and you can use the current date as a dynamic configuration.
- Use the File (Field) Paths module for ALL the options. With this module, you can access all node tokens, such as the date when the node was created or the author. The File field paths module also lets you rename the files.
If you're new to Drupal, I've got a more complete tutorial for you. In my live online walkthrough last week, I walked through all the basics of adding file fields, how to configure private or public download methods, and other essentials. I mentioned the very important modules you should already be using, and I talked about additional modules you can use to improve the process, and some future friendly tips about what is being worked on for media and file management in Drupal 8. You can watch it here. You can also flip through the slides here.
Some good news for Drupal 8!
I'm guessing that Dries's tweet about this issue was sparked most likely by a client issue on our own servers.
Saw a Drupal site crash because Drupal stored hundreds of thousands of files in a single directory. Maybe we should be smarter about that?
— Dries Buytaert (@Dries) November 5, 2013
The good thing is, it prompted a Twitter discussion and an issue now for Drupal 8 to set these defaults to something safe and reasonable. See Files should automatically be uploaded to directories based on a token for more details and to weigh in on the issue.
Come to our live and online training sessions
Check out our training events calendar for public training. In Site Building with Drupal 7, you can learn all the nuts and bolts of building flexible future friendly websites. We'll be adding more dates soon, but please tell us where you'd like to see training next.
More upcoming webinars to prevent Drupal headaches like this!
My live online walk throughs are on Wednesdays. 10 am in California, 1pm Boston time, 6pm in London. It's at 3am Tokyo time, and 5am in Sydney, so if you don't want to waking up at odd hours, you can sign up anyway, and you'll get a notification when the recordings and the slides are available!
