Tag Archives: Linux Sysadmin

PostgreSQL Replication to a Warm-Standby Using WAL Files

Posted on by
Reply

THEORY

Like a good relational database, PostgreSQL maintains a set of transactional log file known as write-ahead-logs (WAL) in the pg_xlog directory.  These logs are written to for every change in the database files and are used to recovery from a crash condition.  If you crash, replay all the WAL files since the last backup and you will be back in business right at the point of failure.

Well, if you have this capability, what about keeping a warm-standby system and feeding it all the WAL files.  If you teach it how to continuously process the incoming write ahead logs from the live system you will have a system ready to go at a moments notice.  When you read about this setup in other places on line the primary server is known as ‘master’ and the secondary the ‘slave’.

NOTE BENE: Both your primary and your secondary need to be running the same major version of the postgreSQL database. Continue reading →

Viewing Your Linux Hardware with DMIDECODE

Posted on by
Reply

I never like opening a running system when I can simply query that system with a simply command for the information needed.  dmidecode is a great tool for polling hardware information in human-readable format.

In its simplest form you will dump all the information to the screen

dmidecode

but that’s a bit much so try running with the -t argument which lets you narrow down the search to the components (bios, system, baseboard, chassis, processor, memory, cache, connector, slot)  So, for instance, if need to learn how much RAM you system can handle:

# dmidecode -t memory
# dmidecode 2.10
SMBIOS 2.7 present.
# SMBIOS implementations newer than version 2.6 are not
# fully supported by this version of dmidecode.

Handle 0x0027, DMI type 16, 23 bytes
Physical Memory Array
    Location: System Board Or Motherboard
    Use: System Memory
    Error Correction Type: Single-bit ECC
    Maximum Capacity: 32 GB
    Error Information Handle: No Error
    Number Of Devices: 4

Enjoy and let me know you you end up using this command.

 

Building a NIS User Add Script

Posted on by
Reply

I have an environment where Solaris provides NIS for all the Solaris and Linux systems.  Every time I add a user I’ve had to alter a number of files and that’s pretty lame.

If you have any questions please ask.

#!/bin/bash
###################
# NewUser.sh creates a new user in the NIS environment and pushes that
# user information out to the server systems.
#
# NewUser.sh v1.0 - jay@zidea.com
#
###################
### Declarations
declare -rx SCRIPT=${0##*/}
declare USERNAME
declare FULLNAME
declare PASSWORD
declare USER_HOME
declare LASTID
declare USERID
### Checks if you have the right privileges
if [ "$USER" = "root" ]
then
#### Collect the variables
echo "" ;echo "" ;echo "" ;echo ""
 printf "%s\n" "Enter the user's name (firstname lastname): "
 echo "" ;echo "" ;echo "" ;echo ""
 read -e FULLNAME
printf "%s\n" "Enter the USERNAME (8 characters or less): "
 echo "" ;echo "" ;echo "" ;echo ""
 read -e USERNAME
# Other variables
USER_HOME="/home/$USERNAME"
 LASTID=`tail -1 /etc/passwd |cut -f3 -d:`
 USERID=`expr $LASTID + 1`
# Checks if the user already exists
 cut -d: -f1 /etc/passwd | grep "$USERNAME" > /dev/null
 OUT=$?
# Test for the account and build the files
 if [ $OUT -eq 0 ];then
 echo >&2 "ERROR: User account: \"$USERNAME\" already exists."
 echo >&2 "ERROR: User account: \"$USERNAME\" already exists." >> "$LOGFILE"
 else
 # Create a new user /usr/sbin/useradd
 /usr/sbin/useradd -u $USERID -d $USER_HOME -g staff -s /bin/bash -c "$FULLNAME" -m $USERNAME
 passwd $USERNAME
 PASSWORD=`grep $USERNAME /etc/shadow | cut -f2 -d:`
 echo $USERNAME:x::::: >> /etc/nis_etc/security/passwd.adjunct
 echo $USERNAME:$PASSWORD:$USERID:10:"$FULLNAME":$USER_HOME:/bin/bash >> /etc/nis_etc/passwd
 echo $USERNAME:$PASSWORD:14785:::::: >> /etc/nis_etc/shadow
# Restart the Yellow Pages (NIS)
 pushd /var/yp
 make
 popd
# Setup the $HOME Directory on svnfiles
 ssh root@home.server.com mkdir -pv /files/$USERNAME
 ssh root@home.server.com chown -R $USERID /files/$USERNAME
 ssh root@home.server.com chgrp -R wheel /files/$USERNAME
echo "The user \"$USERNAME\" has been created."
 fi
 exit 0
else
 echo >&2 "ERROR: You must be a root user to execute this script."
 exit 1
fi

Tuning mySQL – Because by default it’s not even close to tuned.

Posted on by
2

Basic tuning of the mySQL is accomplished in the /etc/my.cnf file. If you want to get all geeky and into this reference the seminal document over on the mysql dev site. This should result in a speed increase in your system.  It certainly has in my system running mySQL 5.x.

The information below is expressed as a set of ratios that begins with your system RAM and then works from there.

innodb_buffer_pool_size = $SYSTEMRAM/2
innodb_additional_mem_pool_size = $innodb_buffer_pool_size/20
innodb_log_file_size = $innodb_buffer_pool_size/4
innodb_log_buffer_size = $innodb_buffer_pool_size/50 or a minimum value of 8MB

Note bene: Changing your log file size can results in a mySQL refusing to start.  Simply remove these files from you mysql data directory and they will be created on the next startup.

Script to Move Database Location – mySQL

Posted on by
Reply

Don’t run this script.  It’s a concept that I haven’t tested and running it is pretty well guaranteed to crash your mysql server.  It’s designed to make the relocation of data faster, but I don’t have time to finish it today.

You should probably use this fellow link because it works… it’s just slower and manual.  Oh, and if you do get a scripting urge, please make this script work properly for me and post it in a comment.  Thanks.

 

USER=root
PASSWORD=yourpassword
DBS="$(mysql --user=$USER --password=$PASSWORD -Bse 'show databases')"
OLDDATA_DIR="/var/lib/mysql"
NEWDATA_DIR="/database/lib/mysql"

mkdir -pv $NEWDATA_DIR

for FILE in ${DBS[@]}; do
        DATABASE=`basename $FILE`
        echo cp -R $OLDDATA_DIR/$DATABASE $NEWDATA_DIR/$DATABASE
done

# Set permissions
chown -R mysql:mysql $NEWDATA_DIR

# Archive the old & link it to the new
mv $OLDDATA_DIR OLDDATA_DIR-old
ln -s $NEWDATA_DIR/$DATABASE $OLDDATA_DIR/$DATABASE

#get_mysql_option mysqld datadir "/database/lib/mysql"
sed -i  's|$OLDDATA_DIR|$NEWDATA_DIR|' /etc/init.d/mysqld
sed -i  's|$OLDDATA_DIR|$NEWDATA_DIR|' /etc/my.cnf

Setting up Apache Log File Rotation

Posted on by
Reply

This how-to walks users through setting up proper log file rotation for a multil-site Apache installation where the log file are broken out by site. I built all this on my own, but forgot about logfile rotation so now the log files just keep growing and growing.  Time to institute a log rotation algorithm.

For the most part when you are working with Unix you will find that the syslog daemon handles how messages are logged in you system, but Apache handles it’s own logs and the details are typically kept in the httpd.conf file.

sudo grep -i 'log' /etc/httpd/conf/httpd.conf /etc/httpd/conf.d/*
# Custom log file locations
LogLevel warn
ErrorLog  /var/www/html/site1.com/log/error.log
CustomLog /var/www/html/site1.com/log/access.log combined
# Custom log file locations
LogLevel warn
ErrorLog  /var/www/html/site2.com/log/error.log
CustomLog /var/www/html/site2.com/log/access.log combined
# Custom log file locations
LogLevel warn
ErrorLog  /var/www/html/site3.com/log/error.log
CustomLog /var/www/html/site3.com/log/access.log combined

So, grepping gives me a listing of logfile locations for each of the sites and as you can see they are all located in different directories.  You probably also noticed that there are logfiles in the con.d directory that I grepped for.  A lot of stuff will want to install there, like phpMyAdmin or webalizer or ssl.conf.  One other note, some installations will have their config files in an apache2 directory. Continue reading →

Mounting a ‘Foreign’ LVM Volume

Posted on by
Reply

First, what do I mean by foreign?  Foreign means mounting the logical volume with an OS that it wasn’t originally installed on.  This could be because you are using KNOPPIX to repair something on the volume, or because you’ve moved the disk to a new location.

The process itself if quite simple, but it would help if you understood how logical volumes work first.  Click here for some nice background

Mounting an LVM Volume

  1. First Identify it with the fdisk command
  2. And find the VolGroup with the pvs command
  3. lvdisplay will show you the Logical Volume
  4. Finally mount it

# fdisk -l
Disk /dev/sda: 500.1 GB, 500107862016 bytes255 heads, 63 sectors/track, 60801 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytes
Device Boot      Start         End      Blocks   Id  System/dev/sda1   *           1          13      104391   83 Linux/dev/sda2              14       60801   488279610   8e  Linux LVM

# pvs  PV         VG         Fmt  Attr PSize   PFree
/dev/sda2  VolGroup00 lvm2 a-   465.66G    0

lvdisplay

Okay, this isn’t finished yet, but I published it so that next time I’m working on this task I’ll complete it.  If you have any suggestions or want to complete this list. let me know.

About Jay Farschman - Jay currently works as a Senior Systems Administrator for an asset management company in Colorado where he works with companies that produce hardware, telecommunications software and financial services.  Jay previously owned a consulting company and provided training and consulting services for three Fortune 500 companies and numerous small businesses where he leveraged Linux to provided exceptional value.

Common Email Error Codes

Posted on by
1

Please help me add to these. If you have more add a comment and I’ll extend the list.  Thanks.

ERROR : “Remote host said: 476 – connection from your host are denied “
REASON : If sending client originates too many requests of the server in too short a time this will result in a lockout of that account for a period of time.  I have seen this happen when I open a huge IMAP account on a new machine.  You can either adjust one or more of the parameters or whitelist the IP Address.

ERROR : “Remote host said: 550 Requested action not taken: mailbox unavailable Giving up on “
REASON : If sending address is invalid,then you will get this message

ERROR : Sorry, I couldn’t find any host named
REASON : Problem with the recipient domain. i.e , The domain name specified is unknown. Either you typed it incorrectly or that domain no longer exists.

ERROR : “Remote host said: 550 abc [at] example [dot] com: Recipient address rejected:” unknown user
REASON : The email address specified by you was incorrect. kindly check the recipient email address.

ERROR : Remote host said: 554 Message type not allowed. UP Email not accepted for policy reasons
REASON : Please add SPF records for your DNS and send the mail.

ERROR : Remote host said:553: Relay access denied
REASON : This error means that the person sending the email was not authorized to use the email server (SMTP) server. In your email-client please enable this option —> my server requires Authentication(In out going server settings)

ERROR : Remote host said 554: Sender address rejected: Access denied
REASON :This error means that the sender address is blacklisted in one of the spam filter lists .

ERROR : 554 #5.5.4 Relaying denied. IP name lookup failed for
REASON : Your IP address does not have a reverse DNS record.

ERROR :Remote host said: 553 sorry, that domain isn’t in my list of allowed rcpthosts
REASON :This error means that the local user ( who is allowed to send mails locally, not to outside domains) is sending mail to the outside world.

ERROR : “Sorry, no mailbox here by that name”
REASON : This is due to invalid email address.please check the email-address and try again.

ERROR : Connected to but greeting failed
REASON : This sometimes occurs when your servers Ip is blacklisted.

ERROR : Connected to but connection died. (#4.4.2) I’m not going to try again; this message has been in the queue too long.
REASON : Recipient server was facing some problem at that time.try to resend the mail again.

ERROR :This message is looping: it already has my Delivered-To line
REASON : problem with the aliases.please check the aliases

ERROR : Sorry, I wasn’t able to establish an SMTP connection. (#4.4.1) I’m not going to try again; this message has been in the queue too long
REASON : If the recipient domain does not have valid mx record, you will get this error.so please contact them and get it rectified.

ERROR : Connected to 192.121.XX.XXX but authentication was rejected (password).“Remote host said: 535 authorization failed”
REASON : This is due to wrong password.please check the password.

ERROR :user [at] example [dot] com:qmail-group: fatal: get ldap group entry: no such object I’m not going to try again; this message has been in the queue too long
REASON :

ERROR : Sorry, user@domain.com is over their allocated Quota
REASON : The recipient’s mailbox is full. Contact them to delete some emails.

ERROR : Remote host said:552 sorry, that message size exceeds my databytes limit
REASON : The size of the attachment is too large.please try to reduce the size of attachment and try to send the mail.

ERROR : Connected to 45.54.XXX.XXX but sender was rejected. Remote host said: 550 Your e-mail was rejected for policy reasons on this gateway. Reasons for rejection may be related to content such as obscene language, graphics, or spam-like characteristics (or) other reputation problems. For sender troubleshooting information, please go to http://postmaster.msn.com. Please note: if you are an end-user please contact your E-mail/Internet Service Provider for assistance.
REASON : Please add SPF record for your DNS.This will solve the problem.

Working with ILOM on a Sun/Oracle System

Posted on by
Reply

Sun(now Oracle) and Fujitsu put together a new remote management system (ILOM) that provides a mild improvement over the older ALOM systems.  This guide will get you started.

#### STARTING STOPPING
The ILOM allows you to manage the system (SYS) with a few simple commands:

-> start /SYS
-> stop /SYS
-> reset /SYS

#### MANAGING AUTO BOOT MODE
-> set /HOST/bootmode script=”setenv auto-boot? false”
-> reset /SYS

#### FORCING A BREAK COMMAND
-> set /HOST send_break_action=break command
-> start /SP/console

#### GRAB A CONSOLE CONNECTION
-> start /SP/console

#### CREATE ILOM USER
-> create /SP/users/<username>
Creating user…
Enter new password: ********
Enter new password again: ********
Created /SP/users/<username>

#However, I want my team to be able to use this system just like they did the old ALOM systems
because it simplifies management for me when I’m on vacation.  Create the user as shown below
to have an old school ALOM experience.

-> create /SP/users/admin role=aucro cli_mode=alom
Creating user…
Enter new password: ********
Enter new password again: ********
Created /SP/users/admin

#### SET A PASSWORD
set /SP/users/root password

#### CHANGING THE IP
-> cd /SP/Network
-> set pendingipaddress=<ip_address>
-> set pendingipdiscovery=static
-> set pendingipnetmask=255.255.255.0
-> set pendingipgateway=<ip_address>
-> set commitpending=true

About Jay Farschman - Jay currently works as a Senior Systems Administrator for an asset management company in Colorado where he works with companies that produce hardware, telecommunications software and financial services.  Jay previously owned a consulting company and provided training and consulting services for three Fortune 500 companies and numerous small businesses where he leveraged Linux to provided exceptional value.

Remote Access in Solaris with the ALOM

Posted on by
Reply

Resetting the ALOM’s network setting is a little un-intuitive, but it not that hard if you remember that Solaris Sunfire systems (v210, v240, v250, etc.) use the scadm interface to address the eeprom/NVRAM.

cd /usr/platform/`uname -i`/sbin
./scadm set netsc_ipaddr 172.16.3.27
./scadm set netsc_ipnetmask 255.255.0.0
./scadm set netsc_ipgateway 172.16.0.8

# Check your work

./scadm show | grep netsc
netsc_tpelinktest="true”
netsc_dhcp="false”
netsc_ipaddr="172.16.3.27″
netsc_ipnetmask="255.255.0.0″
netsc_ipgateway="172.16.0.8″

You really need to have tpelinktest=”true” before proceeding.

# Reset the interface and then look at the settings.

./scadm resetrsc
./scadm shownetwork

About Jay Farschman - Jay currently works as a Senior Systems Administrator for an asset management company in Colorado where he works with companies that produce hardware, telecommunications software and financial services.  Jay previously owned a consulting company and provided training and consulting services for three Fortune 500 companies and numerous small businesses where he leveraged Linux to provided exceptional value.