February 16, 2006

Hard Disk Problem

In the last two months, my server got the following errors occasionally (very infrequent):

> ad0: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=40<UNCORRECTABLE>
LBA=154033791
> ad0: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=40<UNCORRECTABLE>
LBA=154033791
> ad0: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=40<UNCORRECTABLE>
LBA=154033791
> ad0: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=40<UNCORRECTABLE>
LBA=150991359

Apache started to generate fatal errors too since two days ago:
> pid 81205 (httpd), uid 80: exited on signal 10
> pid 82339 (httpd), uid 80: exited on signal 10
> pid 81204 (httpd), uid 80: exited on signal 10
> pid 88426 (httpd), uid 80: exited on signal 10
> pid 89324 (httpd), uid 80: exited on signal 10
> pid 81207 (httpd), uid 80: exited on signal 10
> pid 88654 (httpd), uid 80: exited on signal 10

Combined these two errors, it's very likely a hard disk problem. I contacted my ISP (LayeredTech), the tech support confirmed: "DMA errors with the UNCORRECTABLE code are almost always evidence of a failing drive."

He suggested a system reload on a new hard disk. But rebuilding the system with a dozen websites might take a few hours or even longer, I really want to avoid such a down time. I decided to order a new server and cancel the current one after finish the transfer. LayeredTech is very understanding and cooperative with prompt response, I am very satisfied.

Posted by FreeBSD Newbie at 10:45 PM | Comments (0)

February 10, 2006

Update AwStats with Cron Job

In the next few days, I will try to automate the server maintenance as much as possible - ideally, I don't have to logon the server for months if I don't want to. Summer is coming :-).

I know little about shell scripts, let me start from an easy one - AwStats.

Name the update script as "awstats_update_all.sh" with the following content:
#!/bin/sh
cd /usr/local/www/awstats/tools/
./awstats_updateall.pl now -awstatsprog=/usr/local/www/awstats/cgi-bin/awstats.pl

The script is also an ideal place to syncronize the server time, ntpd is overkill with annoying output.
/usr/sbin/ntpdate pool.ntp.org

I also want to know the execution time, so my complete version is:
#!/bin/sh
cd /usr/local/www/awstats/tools/
date
./awstats_updateall.pl now -awstatsprog=/usr/local/www/awstats/cgi-bin/awstats.pl
date
/usr/sbin/ntpdate pool.ntp.org

Put it in cron job, update daily at 2:01am:
01 02 * * * /home/some_account/awstats_update_all.sh

Posted by FreeBSD Newbie at 01:05 AM | Comments (1)

January 19, 2006

301 Redirect

Just want to post a follow-up to my previous post. After I updated the domain configurations, all my sites have been re-indexed by Google. I strongly suggest you configure your domains this way.

Posted by FreeBSD Newbie at 12:45 AM | Comments (0)

January 13, 2006

Apache and Google

This post is more about Google, but I think it's very important and I should mention it.

Let me use this blog as an example to explain the problem. This blog can be accessed by typing freebsdblog.org or www.freebsdblog.org in your browser, there is no any difference to you. But Google treats them as two different websites, since they have exactly the same content, Google may dump out one of them due to duplicated content penalty. If Google bans the one you have been promoting, that would be a big problem.

This may sound very silly, but it has been discussed for years and reported by many webmasters. Recently, my sites also had some problems like this, although I'm not very sure about the reasons. If you try Yahoo, Google, MSN or FreeBSD, all of the big guys redirect you to the www version with permanent redirection (HTTP 301). Anyway, I strongly suggest you configure your domains this way, by which you will only benefit from it:

Continue reading "Apache and Google"

Posted by FreeBSD Newbie at 05:36 PM | Comments (1)

August 18, 2005

Install Awstats 6.5 Without Ports

Awstats in ports is still marked "forbidden". I like to check out the web logs when I'm bored. I decide to install Awstats 6.5 in the traditional way instead of waiting for the port release. Since I have used Awstats before, the domain configuration files, .htaccess and password files for directory protection are still there, "make deinstall" doesn't delete them. The installation only took several minutes.

# cd /usr/local/www
# wget http://awstats.sourceforge.net/files/awstats-6.5.tar.gz
# tar xvf awstats-6.5.tar.gz
# chmod +rx awstats-6.5
# cd tools
# ./awstats_configure.pl
# apachectl graceful

Now, Awstats is ready without directory protection. If you use "online update mode" for Awstats like me, protecting the stats is recommended.

Please note that installing from ports is preferred whenever possible, this is only my temporary solution.

Posted by FreeBSD Newbie at 12:31 PM | Comments (0)

July 31, 2005

Httpd Exited on Signal 11 - Caused by Buggy Script

My security run output of today contains lots of error messages like:
pid 90742 (httpd), uid 80: exited on signal 11
pid 90896 (httpd), uid 80: exited on signal 11

It looks like a serious problem, then I searched the Internet immediately, here is a helpful discussion. Although there are no definite answers, the possible reasons can be summarized as:
1. Hardware problems, the most common one.
2. Vulnerable version of Apache, PHP or other Apache modules.
3. Buggy scripts.

Regarding my server
1. Hardware problems
This server has been running well for several months, unlikely to be the case.

2. Vulnerable version of Apache or its modules
Normally I can ignore this since I monitor the packages closely with portaudit. But recently Apache did have a known vulnerability. I ignored the upgrade because it shouldn't affect my installation.

This only affects installations where Apache is used as HTTP proxy in combination with the following web servers.

3. Buggy scripts
I did install mambo yesterday and it has a very weird problem, admin login doesn't work. There are many similar bug reports without solutions on their support forum, which is very unusual for a popular CMS, IMO. I debugged the script as I wrote in a previous post, it turned out that the login script ended at a PHP function "session_start()", it didn't give any error message, just like calling "die()". Then I made a test, signal 11 error happened every time I made a login.

Now the problem is clear, I removed mambo.

Posted by FreeBSD Newbie at 02:05 PM | Comments (1)

July 24, 2005

Hide Apache and PHP Information in HTTP Headers

By default, Apache will send version and modules information (e.g., mod_php, mod_perl, mod_ssl) in every HTTP header. You can check it with a HTTP header tool. For example, the header of this blog:
Server: Apache/1.3.33 (Unix) PHP/4.3.11

To hide the information, config Apache:
ServerTokens ProductOnly
ServerSignature Off

The header changed to:
Server: Apache

But for a PHP powered website, PHP engine will add its information to the headers regardless of Apache configuration:
Server: Apache
X-Powered-By: PHP/4.3.11

To avoid this, turn off expose_php in php_ini:
expose_php = Off

Posted by FreeBSD Newbie at 04:51 PM | Comments (0)

June 18, 2005

Performance Tuning

I have been considering to upgrade my 10 MB uplink, but after read the system tuning in the handbook, I realized something else was more important than the bandwidth limit, especially kern.ipc.somaxconn:

The default value 128 is typically too low for robust handling of new connections in a heavily loaded web server environment. For such environments, it is recommended to increase this value to 1024 or higher.

I raised kern.ipc.somaxconn to 1024 with the sysctl command:
# sysctl kern.ipc.somaxconn=1024

To avoid running this command every time the server reboots, I also added the following line to /etc/sysctl.conf:
kern.ipc.somaxconn=1024

Apache could be a bottleneck too. I also modified Apache configuration and did a graceful restart:
MaxKeepAliveRequests 200 (from 100)
MaxClients 300 (from 150)

Posted by FreeBSD Newbie at 01:56 AM | Comments (1)

June 10, 2005

Move Apache Log Files

When I checked emails with Outlook Express today, all POP3 accounts on my two servers gave me an error message "Invalid username or password", which really freaked me out. Later I found it's because /var file system used up again. My ISP assigned 4GB to /var by default, I thought it's enough after I linked /var/db to other place, obviously I underestimated Apache log files - which can consume 4GB in a couple of weeks. To avoid any possible strange errors, also to make /var/log cleaner (I placed Apache logs file under /var/log/), I decide to move out Apache log files too.

This is quite simple though, make a folder /home/apachelog/, link it to /var/log/www. In order to keep it more flexible, I didn't use /home/apachelog/ directly in httpd.conf file.

Detailed Steps
1. # mkdir /home/apachelog
2. # ln -s /home/apachelog /var/log/www
3. Make a global replace in Apache configuration file.
vi command: :%s/\/var\/log/\/var\/log\/www/g
4. Restart Apache
5. Modify Awstats configuration files for every domain.

By the way, I'm reading the book "Mastering FreeBSD and OpenBSD Security", only finished a small part so far, but I find it's a great book, I hope I can learn some practical tips soon.

Posted by FreeBSD Newbie at 11:15 PM | Comments (0)

June 01, 2005

More About Apache Log File Rotation

As I wrote earlier, I remove old Apache log files by deleting all of them and doing a graceful restart, this is also what Apache documentations suggested, everything works well except that Apache doesn't release the disk space used by the deleted log files. As a result, "du" command shows that /var was using 100% disk space, but if I run "df -ma" under /var, it's only using 23 MB. Postfix can't receive any emails since disk space was used up, which gave an error message like: "Out: 452 Insufficient system storage". A non-graceful Apache restart solved the problem.

In Summary
"apachectl graceful" doesn't work 100% correctly, it holds the disk space used by deleted log files. I didn't test how it works if the new log file uses different file name.

Posted by FreeBSD Newbie at 05:34 AM | Comments (0)

April 09, 2005

Protect Directory

How to protect a directory under Apache?

This can be done with .htaccess, visitors must input username and password before they can access the protected directory.

Step 1.
Create the file ".htaccess" with the following content under the directory you want to protect, you can specify the password in any place as long as Apache can access it.

AuthUserFile /home/account/passwd/.htpasswd
AuthGroupFile /dev/null
AuthName "Title"
AuthType Basic

<Limit GET POST>
require valid-user
</Limit>

Step 2.
Generate the password file ".htpasswd" under the directory /home/account/passwd/ with the following command:
# htpasswd -c .htpasswd user_name

You will be reminded to input password for the user "user_name", this command can be called multi times to generate multi users.

Step 3.
This step may not be necessary. If there is an entry about the directory you want to protect in Apache configuration file, make sure AllowOverride has the option authconfig.

For example, when protecting Awstats pages, the Apache configuration file should be modified as following:

<Directory "/usr/local/etc/awstats/wwwroot">
Options None
AllowOverride authconfig
Order allow,deny
Allow from all
</Directory>

Restart Apache
# /usr/local/sbin/apachectl restart

Posted by FreeBSD Newbie at 11:54 PM | Comments (1)

January 21, 2005

Install Awstats on FreeBSD

Warning: If you are using Awstats 6.2 or earlier version, your server is at great risk, update it now! More information.

I have separated the log files for all virtual hosts, now it's time to install a log analyzer. I used to use offline software, but as the website growing, a faster and more convenient way is a online program.

1. Install Awstats from ports
# cd /usr/ports/www/awstats
# make install
It seems that "make install" only did a download.

2. Config Awstats
Find the file "config.pl", it's under /usr/ports/www/awstats/work/awstats-6.1/tools on my server, run
# ./config.pl
It suggests that the config script should run from /usr/local/awstats, then I copied the whole directory to /usr/local/etc/awstats/
# cp -Rv /usr/ports/www/awstats/work/awstats-6.1/ /usr/local/etc/awstats
(Note: the directory is different for different Awstats versions.)

# chmod +x /usr/local/etc/awstats
(Give Apache permission)

# /usr/local/etc/awstats/tools/configure.pl
This script will ask you a few questions to config Apache, follow the instructions and a config file for this domain will be created. It's better to save the output for reference, many useful information.

3. Modify generated Awstats config file
The config file is generated under /etc/awstats/, named like awstats.www.domain.com.conf
It's necessary to modify this file, the important parameters are:
LogFile="/var/log/domain.com-access_log"
DirData="/var/log/awstats"
(You need to create this directory)

4. Restart Apache
# /usr/local/sbin/apachectl restart

5. Generate stats
This step can be done from the web if you enable AllowToUpdateStatsFromBrowser in the file awstats.www.domain.com.conf.
# /usr/local/etc/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.domain.com
If everything goes well, it will dispaly something like:

Update for config "/etc/awstats/awstats.www.domain.com.conf"
With data in log file "/var/log/domain.com-access_log"...
Phase 1 : First bypass old records, searching new record...
Direct access after last parsed record (after line 74)
Jumped lines in file: 74
Found 74 already parsed records.
Parsed lines in file: 0
Found 0 dropped records,
Found 0 corrupted records,
Found 0 old records,
Found 3 new qualified records.

6. Test the result
Visit http://www.domain.com/awstats/awstats.pl?config=www.domain.com
Get "permission error"? Check if Apache can read the directory /usr/local/etc/awstats/.

7. Config other domains
Fortunately, you don't have to repeat the config process for all virutal hosts, copy the previous config file and modify the parameters as step 3.

8. Add to cron job
# crontab -e
Insert the following line, Awstats will update at 00:01 everyday.
01 0 * * * /usr/local/etc/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.domain.com
02 0 * * * /usr/local/etc/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.otherdomain.com
(more info about cron job)

Awstats has a command to update all domains, but I wasn't able to run it.
# /usr/local/etc/awstats/tools/awstats_updateall.pl now

Awstats also supports update from the web, I prefer this way - generate the stats only when I need it, add the following line in the file awstats.www.domain.com.conf.

AllowToUpdateStatsFromBrowser=1

9. Protect the Awstats Report
You might want to protect the awstats report, refer to "how to protect a directory under Apache". Note that you must modify Apache configuration file.

Posted by FreeBSD Newbie at 07:22 AM | Comments (2)

Separate Log Files for Virutal Host

To have dedicated log file for each virtual host, add the following directive to VirtualHost directive in Apache config file: (FreeBSD)

ErrorLog /var/log/error_log_file_name
CustomLog /var/log/log_filename combined

The complete VirtualHost entry looks like:
<VirtualHost 1.2.3.4>
ServerAlias www.domain.com domain.com
ServerAdmin admin@domain.com
DocumentRoot /home/domain/www
ServerName www.domain.com
ErrorLog /var/log/error_log_file_name
CustomLog /var/log/log_filename combined
</VirtualHost>

Make sure to check the config file before restarting Apache:
# /usr/local/sbin/httpd -t

Posted by FreeBSD Newbie at 01:45 AM | Comments (0)

January 13, 2005

Mod_rewrite

mod_rewrite is enabled by default after installation, make sure the following two lines are present in Apache configuration file (/usr/local/etc/apache/httpd.conf).
LoadModule rewrite_module libexec/apache/mod_rewrite.so
AddModule mod_rewrite.c

A quick command to check this:
# cat /usr/local/etc/apache/httpd.conf | grep mod_rewrite

If you want to use url rewrite in .htaccess file, you must modify httpd.conf.
Find
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
Replace with
<Directory />
Options All
AllowOverride All
</Directory>

Posted by FreeBSD Newbie at 04:38 AM | Comments (0)