Tag Archives: tuning

Tuning mySQL – Because by default it’s not even close to tuned.

Posted on by
2

Basic tuning of the mySQL is accomplished in the /etc/my.cnf file. If you want to get all geeky and into this reference the seminal document over on the mysql dev site. This should result in a speed increase in your system.  It certainly has in my system running mySQL 5.x.

The information below is expressed as a set of ratios that begins with your system RAM and then works from there.

innodb_buffer_pool_size = $SYSTEMRAM/2
innodb_additional_mem_pool_size = $innodb_buffer_pool_size/20
innodb_log_file_size = $innodb_buffer_pool_size/4
innodb_log_buffer_size = $innodb_buffer_pool_size/50 or a minimum value of 8MB

Note bene: Changing your log file size can results in a mySQL refusing to start.  Simply remove these files from you mysql data directory and they will be created on the next startup.

Setting up Apache Log File Rotation

Posted on by
Reply

This how-to walks users through setting up proper log file rotation for a multil-site Apache installation where the log file are broken out by site. I built all this on my own, but forgot about logfile rotation so now the log files just keep growing and growing.  Time to institute a log rotation algorithm.

For the most part when you are working with Unix you will find that the syslog daemon handles how messages are logged in you system, but Apache handles it’s own logs and the details are typically kept in the httpd.conf file.

sudo grep -i 'log' /etc/httpd/conf/httpd.conf /etc/httpd/conf.d/*
# Custom log file locations
LogLevel warn
ErrorLog  /var/www/html/site1.com/log/error.log
CustomLog /var/www/html/site1.com/log/access.log combined
# Custom log file locations
LogLevel warn
ErrorLog  /var/www/html/site2.com/log/error.log
CustomLog /var/www/html/site2.com/log/access.log combined
# Custom log file locations
LogLevel warn
ErrorLog  /var/www/html/site3.com/log/error.log
CustomLog /var/www/html/site3.com/log/access.log combined

So, grepping gives me a listing of logfile locations for each of the sites and as you can see they are all located in different directories.  You probably also noticed that there are logfiles in the con.d directory that I grepped for.  A lot of stuff will want to install there, like phpMyAdmin or webalizer or ssl.conf.  One other note, some installations will have their config files in an apache2 directory. Continue reading →

Nmap with three commands

Posted on by
Reply

Network Mapper is an essential tool for every SysAdmin.  You need to probe around the network to make sure people haven’t left ports open in a haphazard manner.   These commands should get you ther.

Ping sweep and a reverse DNS shows you who is on the logical network with you:

nmap -sP 10.1.1.0/24

-sS does a bit of scanning for ports

nmap -sS 10.1.1.23

Or you can set off all the alarms on the network with the following pots scan of everything.

nmap -O 10.1.1.0/24

About Jay Farschman - Jay currently works as a Senior Systems Administrator for an asset management company in Colorado where he works with companies that produce hardware, telecommunications software and financial services.  Jay previously owned a consulting company and provided training and consulting services for three Fortune 500 companies and numerous small businesses where he leveraged Linux to provided exceptional value.

 

Measuring Drive Speed in Linux

Posted on by
Reply

Benchmarking drive speed is made a lot easier in Linux by the presence of a tool, hdparm.  On IDE style drives and ATA running with the arbitrator they produce pretty good results:

#hdparm -t /partition/totest

Some Original Results prior to messing with the strip sizes:

PowerEdge 2950 -> 20 to 90 mb/sec
PowerEdge 6800 -> 25 to 30 mb/sec

hdparm _only_ works when the Integrated Drive Electronics (IDE) are talking directly to the system over the AT Attachment (ATA) arbitrator.  So see if you are running the libata module by using the lsmod command. If you see it is running… but remember it could be running for one set of diskc and not another.

YOUR BEST BET: Use “modinfo -p” on the vendor’s SCSI driver to see what options are supported at load time.

Other note to measure Random access times… use seeker.c and compile it. it will run a series of random seeks and report the times.  After some rebuilds I have done some benchmarking along with some changes to increase RAID5 stripe sizes in the PE2950 systems:

# New Results After Increasing the Strip Size
Backup Server PowerEdge 2950 – 319.06 MB/sec
PowerEdge 6800 - 92.16 MB/sec

This is by no means a comprehensive discussion of benchmarking, but it’s a good way to start the conversation and it only takes a few moments

About Jay Farschman - Jay currently works as a Senior Systems Administrator for an asset management company in Colorado where he works with companies that produce hardware, telecommunications software and financial services.  Jay previously owned a consulting company and provided training and consulting services for three Fortune 500 companies and numerous small businesses where he leveraged Linux to provided exceptional value.

Linux Performance Monitoring

Posted on by
Reply

There are really two main possibilities that may be limiting your performance in Linux. This document is written to take you through the process of discovering where the bottleneck is on Redhat or CentOS, but it should work on most other versions of Linux as well.

A GOOD PLACE TO START – THE TOP
The top program provides a dynamic real-time view of a running system. It can display system summary information as well as a list of tasks currently being managed by the Linux kernel. The top command monitors CPU utilization, process statistics, and memory utilization. The top section contains information related to overall system status – uptime, load average, process counts, CPU status, and utilization statistics for both memory and swap space.

WANT TO SEE THE INDIVIDUAL PROCESSES?
You can use the command mpstat -P ALL, to get this output, but I like a new tool called HTOP

yum install htop

Then just run it with ‘htop’ on the command line. You will see all the CPUs at the top of the screen.

NOT AS USEFUL
Try SAR. The sar command writes to standard output the contents of selected cumulative activity counters in the operating system. The accounting system, based on the values in the count and interval parameters.

I don’t like sar as much, but you can build a log file by redirecting the output to

nohup sar -o output.file 12 8 >/dev/null 2>&1 &

WHO ARE THE CPU EATERS

ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10

WHO ARE THE CPU EATERS

What kind of throughput am I getting on the ethernet connection?

yum install iptraf

INPUT/OUTPUT STATISTICS
The iostat command kind of rocks when checking for disk and partition usage.

iostat -d -x 2

will cause a display of juicy data to appear. The command above is going to display the data every two seconds so you may want to resize the window to handle the refresh gracefully.

What does it all mean?
Well, the last three columns are the most important. And note: as %util approaches 100% the device or partition is approaching saturation.

  • rrqm/s : The number of read requests merged per second that were queued to the hard disk
  • wrqm/s : The number of write requests merged per second that were queued to the hard disk
  • r/s : The number of read requests per second
  • w/s : The number of write requests per second
  • rsec/s : The number of sectors read from the hard disk per second
  • wsec/s : The number of sectors written to the hard disk per second
  • avgrq-sz : The average size (in sectors) of the requests that were issued to the device.
  • avgqu-sz : The average queue length of the requests that were issued to the device
  • await : The average time (in milliseconds) for I/O requests issued to the device to be served. This includes the time spent by the requests in queue and the time spent servicing them.
  • svctm : The average service time (in milliseconds) for I/O requests that were issued to the device
  • %util : Percentage of CPU time during which I/O requests were issued to the device (bandwidth utilization for the device). Device saturation occurs when this value is close to 100%.

PUTTING IT ALL TOGETHER
Realistically, if you call up htop you can monitor the CPU usage. Call up iostat to see the disk i/o. That will give you are real time reading of what is happening.

 

About Jay Farschman - Jay currently works as a Senior Systems Administrator for an asset management company in Colorado where he works with companies that produce hardware, telecommunications software and financial services.  Jay previously owned a consulting company and provided training and consulting services for three Fortune 500 companies and numerous small businesses where he leveraged Linux to provided exceptional value.