The story:
I was getting session file write errors (write failed: No space left on device (28) ) in wp admin, i checked /tmp folder but it was not full, only 60% percent was used. So not being able to pinpoint what the problem was, i thought a reboot would be a good idea. But everything went straight to hell from there. The reboot returned with a “Error establishing a database connection“, so i went on to restart mysql. Needless to say it failed. While i was at it, i decided to upgrade mysql to 5.5 – but i dont know if that made matters worse, or effected it in any way.
I kept getting ‘unrecognized service’ errors for mysql and mysqld in my restart attempts. Which made absolutely no sense because i was damn sure mysql was installed since i had a properly running server 10 minutes ago.
Despite the fact that i was unable to start mysql, i was still able to access phpmyadmin. My tables seemed to be corrupt but i could not neither check or repair them. They were marked as #Unknown Storage Engine innodb & in use.
Turns out; all the problem stemmed from /tmp running out of ‘inodes‘.But we’ll get to that.
Before i figured that out, i tried a lot of things. I removed(renamed) the host.domain.pid & ib_logfile1, ib_logfile2 files in var/lib/mysql. At this point i started to get The server quit without updating PID file errors. Google searches suggested restarting mysqld and it’d be taken care of, but i wasnt able to do that already. When i figured i was out of inodes by doing a #df -i as in the picture, i wondered what wore them out and found out it was eaccelerator. So i’ve cleaned up /tmp/eaccelerator folder and moved it to /var/cache/eaccelerator, and defined it in php.ini as eaccelerator.cache_dir=”/tmp/eaccelerator” . Cleaning out the inodes, i was able to start mysql now. But my troubles wasnt done yet.
I had a highly customized my.cnf and thought something might be conflicting. It didnt make any sense for a conflict to appear out of the blue, but to my surprise, something was. But i was already tired by now and definitely wasnt in the mood to go over tens of my.cnf lines to pinpoint the problematic line. So i chose one of the defaults inside /usr/share/mysql/ . Dont worry, these mysql configs are specifically generated for your server so you should be fine, just pick one that suits you (i picked the most suitable one for me: #DESCR: 4GB RAM, few connections, heavy queries).
I don’t know if it was the 5.5 mysql upgrade or something else i did along the way, but when i try to go in phpmyadmin, i was rejected with a #2002 Cannot log in to the MySQL server while trying to connect to phpmyadmin & now i was welcomed by a shocking “wordpress install” page. Oddly enough, i had to reconfigure my phpmyadmin config & accordingly wp config. Had to change connection type of phpmyadmin from tcp to socket and change “localhost” to “127.0.0.1” and again localhost to 127.0.0.1 in wp-config to reflect the changes.
Yeah, so that’s all, those are what i ended up having to do. Maybe if i noticed i was out of inodes in the beginning, i wouldnt have to change a thing (other than the eaccelerator cache folder), eh, what you gonna do.. live & learn.
Problems:
If your WordPress defaults-reverts to install screen
if you get wp error: Error establishing a database connection
If your tables are marked #Unknown Storage Engine innodb or ‘in use‘ in phpmyadmin,
if you cannot start mysql
If you get mysqld: unrecognized service or mysql: unrecognized service when trying to start/restart them.
If you get #2002 Cannot log in to the MySQL server while trying to connect to phpmyadmin.
If you get WordPress database error: [Can’t create/write to file ‘/tmp/#sql_xxxx.MYI’ (Errcode: 28)]
Causes:
Your /tmp is full (the fact that it has unused space doesnt mean it’s NOT FULL. It might have run out of inodes – read above)
Your phpmyadmin configuration is wrong.
Your wp-config is wrong.
Solutions:
Make sure your /tmp folder is not full BOTH in terms of space & inodes.
If you are sure of that, check your my.cnf and try loading one of the default ones, see if that solves it.
Check your phpmyadmin config & your wp-config complies with it.