Battling with Wordpress on the field of file permissions

This post is about two problems with fresh installations of the latest version of WordPress (3.3.1) and how I solved them, with a bit of bitching thrown in. I hope they help somebody.

Broken Media uploads

The first problem concerned uploading files to the Media Library. After selecting a file, a progress bar ran from 0% to 100% without problems, afterwhich the following error was displayed:

Unable to create directory /foo/bar/baz/wp-content/uploads/2012/03. Is its parent directory writable by the server?

Now, this looks on the surface like a cut-and-dried file permissions problem. I fully expected to be able to use chmod to fix this in a minute. Indeed, if you Google this error message you will find plenty of people reporting this problem, being told to change their wp-content directory's permissions (alarmingly, people often advise people to change the permissions to 777!), and reporting back that this, indeed worked.

It didn't work for me. I chowned the wp-content directory and everything in it to _httpd, and chgrped it to _httpd too. I even tried setting the permissions to 777. No change. To convince myself I wasn't crazy, I changed the _httpd user's shell from /sbin/nologin to /bin/ksh and actually became that user, changed to wp-content and used mkdir and touch to convince myself that, yes, WordPress truly and absolutely had permission to create the upload directory.

Back to Google, then. A lot of people recommended checking out the "Store uploads in this folder" setting in Settigs > Media. Apparently people sometimes set this to /wp-content/uploads instead of just wp-content/uploads, the leading / changing it into an absolute rather than relative path. And apparently people often migrate from one host to another and bring a server-specific value of this setting with them, which causes failures on the new server because WordPress is trying to create the upload directories in a location which doesn't exist. In my case, however, there was nothing in this field, so WP was correctly defaulting to wp-contents/uploads. The error message I was getting involved a perfectly valid path, anyway, so there's no way it could have been this.

Google a little further. Somebody suggested that this was caused by running PHP in Safe Mode, and that turning Safe Mode off would fix the problem. I was (and am) running PHP in Safe Mode, but I didn't see how this could possibly cause this problem. I had the safe_mode_gid option on, and all of the files in my WP install are in the _httpd group, so it absolutely shouldn't have caused a problem (besides, SM stops PHP from executing certain .php files based on where they are in the filesystem and who owns them - it shouldn't affect PHP's ability to make a new directory at all, as I understand it).

Nevertheless, I tried disabling Safe Mode and lo and behold, it worked! Now, apparently unlike a lot of people in the WP community, I'm not remotely happy about disabling security features the second they cause a problem. I wanted to keep Safe Mode on and have Media uploads work.

What ended up working was chgrping the wp-admin directory to _httpd as well. This directory contains the media-new.php and media-upload.php files which are involved in Media uploads. I already had these files in the _httpd group, but apparently this alone was not enough - their parent directory must also be in the group. I don't know why this should be the case, but at least I got things working.

Forced to use FTP to install themes or plugins

The second problem relates to installing themes or plugins from inside WordPress' administrative interface. Whenever I would try to do this, WP would ask me for FTP details so it could install the relevant files. This baffled me, since WP has direct access to the relevant part of the filesystem, so why on Earth should FTP be involved?

Google this problem, and you'll find plenty of people telling you how to "fix" this problem by putting your FTP details into your wp-config.php file so WP can just use them without asking in future. Of course, this is not actually solving the problem at all, it's just brushing it under the rug. You shouldn't need to use FTP at all if it's possible to get at the filesystem directly. Doing so is positively Rube Goldbergesque.

This page, by Chris Abernethy, has the true answer. The story is that WP asks for FTP details if it thinks (emphasis on thinks) that it doesn't have permission to write to the filesystem directly. Now, once again, I was absolutely sure that WP did have permission to write to the filesystem directly, and had verified this by creating directories and files in the appropriate place manually as the _httpd user. However, Chris' page explains why this might not be enough. WP uses a weird and round-about way to determine if it has appropriate permissions. WP creates a temporary file somewhere, checks which user owns that file, checks who owns the file currently being executed, and compares the two users. If they aren't the same, it assumes it isn't allowed to write to the filesystem directly and doesn't even try. As such, it's not enough that WP has permission to read the scripts that deal with theme and plugin installation, it has to actually own those scripts.

I think this is stupid (and I'll elaborate on it later), but it seemed easy enough to fix. I chowned the relevant script to _httpd, but this didn't fix anything. I spent some time flailing around, randomly chown, chgrping and chmodding things, but nothing seemed to work. Going by the snippet of code Chris posted, which shows how WP actually implements this bizarre little check, I had a look at the code WP uses to create the temporary file, and then at the code it uses to determine which directory to create that file in. As far as I could tell, WP was probably trying to create a file in wp-content/tmp, a directory which didn't actually exist. I created it, and set its user and group to _httpd, but still no luck.

In the end, I took the easy way out: following the advice of a comment on Chris' post, I added "define('FS_METHOD', 'direct');" to my wp-config.php. This short-circuits the bizarre little file ownership test and just tries to access the filesystem directly no matter what. And you know what? It just worked. Well, technically first it downloaded a theme but then complained it couldn't uncompress it because the zlib extension was missing. So I installed that extension. Then it complained that it couldn't create a directory - but for very good reason, because it actually couldn't create that directory, according to the file permissions. I granted it that permission in a straightforward and common sense manner, and then it worked just fine.

Commentary

I find it really frustrating that WP downloaded the theme immediately after I explicitly set FS_METHOD to direct, and obviously had zero permission related trouble in doing so, but when left to its own devices stubbornly insisted that it didn't have permission to do so. I have absolutely no idea why WP uses such a circuitious way to test for filesystem permission. There must be a PHP function to directly ascertain the answer to "do I have permission to write to this exact location right here?". Maybe there is one and it doesn't work well on Windows or other non-POSIX servers, so the WP folks tried to work around it and weren't too smart about it. Maybe there's some other good reason for it. But the simple fact of the matter is that their odd little test, as it currently stands, is undeniably broken: it thinks that WP doesn't have permission to do something it absolutely does. That should be considered a bug.

This whole thing reminds me of a common saying in the Python community: "it's easier to ask for forgiveness than permission". This means: don't bother trying to carefully ascertain if you have permission to do something, and then only go ahead if you do. It means just try and do the damn thing, and if somebody yells at you (in this case, the operating system), deal with the problem then. I feel like this applies very well to this case. There's really no penalty to trying to create a file or directory where you aren't allowed to. You'll just get an error back and then you can try something else, no harm done. If WP took this approach to installing themes and plugins, I wouldn't have had a problem in the first place.

Earlier I said that I thought it was stupid that WP apparently requires ownership of some of some of its scripts, rather than just read-access (although even that wasn't sufficient in my case). This is because it makes it impossible to use what I think is the One True Way to assign permissions to files in a website. I like using this scheme for everything inside the document root of a webserver:

• All files and directories are owned by my everyday user account
• All files and directories belong to a group which contains only a single user, the user which the webserver runs as
• All files and directories are given a permission of 740 by default
• Files and directories are then given permissions of 760 or 770 on an as-needed basis

This means that:

• Root can do anything to anything - they're root
• I can do anything to anything, which makes sense because they're my files and when I ssh into my webserver I don't want to be using su or sudo just to make changes to my website
• The webserver can read everything, so as to be able to serve it, and can write or execute only those things that it absolutely needs to
• Anybody who isn't root, me or the webserver can't do a damn thing

This set up just makes sense to me. It gives out as little power as necessary to as few users as necessary to ge the job done, and that's it. This is analogus to the "default deny" style of writing firewall rules. It seems like good security practice.

If I am forced to make some WP scripts owned by the webserver user, then the only way I can make sure I and I alone can still work on those scripts using my everyday user account is to set the group of that script to some group that contains just my user, which seems silly.

I'm glad I was able to solve both these problems in the end, but Googling around before doing so has left me with a distinct impression that, to paraphrase Zed Shaw, WordPress is a bit of a ghetto.

I can't really say all that much about the WP developers themselves, or the codebase overall (although the few snippets of code I looked at today obviously didn't thrill me). But with regards to the community? Yikes. It's just loaded with people handing out advice which they often obviously don't really understand, and/or is just really bad advice. People are all too happy to recommend things like setting file or directory permissions to 777, or chowning your entire WP installation to your webserver user (usually thus giving every part of WP write access to every other part of WP, and possibly to static sites hosted on the same server) or disabling PHP Safe Mode, and people are all too happy to do those things as long as they solve a problem. Very rarely have I seen people call anyone out on advising these sorts of things. Plenty of people also seem to find nothing unusual or wrong about just relying on FTP to install themes and plugins to their WP install, despite how truly bizarre this is. All up, I was reminded of this line from the NetBSD webpage:

Some systems seem to have the philosophy of "If it works, it's right". In that light NetBSD could be described as "It doesn't work unless it's right".

I get the impression that "if it works, it's right" is a prevalent attitude in the WordPress world.