RHEL Missing in Action?

| No Comments | No TrackBacks
Do you get this error message?

[Error: failed to retrieve repodata/filelists.xml.gz from rhel-x86_64-server-5
error was [Errno 14] HTTP Error 404: Not Found]


Then try the following commands::

# sudo rhn-profile-sync
# sudo yum clean all
# sudo yum update -y

MySQL Replication

| No Comments | No TrackBacks

How To Set Up Database Replication In MySQL

Version 1.1
Author: Falko Timme <ft [at] falkotimme [dot] com>
Last edited: 01/14/2006

This tutorial describes how to set up database replication in MySQL. MySQL replication allows you to have an exact copy of a database from a master server on another server (slave), and all updates to the database on the master server are immediately replicated to the database on the slave server so that both databases are in sync. This is not a backup policy because an accidentally issued DELETE command will also be carried out on the slave; but replication can help protect against hardware failures though.

In this tutorial I will show how to replicate the database exampledb from the master with the IP address192.168.0.100 to a slave. Both systems (master and slave) are running Debian Sarge; however, the configuration should apply to almost all distributions with little or no modification.

Both systems have MySQL installed, and the database exampledb with tables and data is already existing on the master, but not on the slave.

I want to say first that this is not the only way of setting up such a system. There are many ways of achieving this goal but this is the way I take. I do not issue any guarantee that this will work for you!

1 Configure The Master

First we have to edit /etc/mysql/my.cnf. We have to enable networking for MySQL, and MySQL should listen on all IP addresses, therefore we comment out these lines (if existant):

#skip-networking
#bind-address = 127.0.0.1

Furthermore we have to tell MySQL for which database it should write logs (these logs are used by the slave to see what has changed on the master), which log file it should use, and we have to specify that this MySQL server is the master. We want to replicate the database exampledb, so we put the following lines into /etc/mysql/my.cnf:

log-bin = /var/log/mysql/mysql-bin.log
binlog-do-db=exampledb
server-id=1

Then we restart MySQL:

/etc/init.d/mysql restart

Then we log into the MySQL database as root and create a user with replication privileges:

mysql -u root -p
Enter password:

Now we are on the MySQL shell.

GRANT REPLICATION SLAVE ON *.* TO 'slave_user'@'%' IDENTIFIED BY '<some_password>';(Replace <some_password> with a real password!) 
FLUSH PRIVILEGES;

Next (still on the MySQL shell) do this:

USE exampledb;
FLUSH TABLES WITH READ LOCK;
SHOW MASTER STATUS;

The last command will show something like this:

+---------------+----------+--------------+------------------+
| File | Position | Binlog_do_db | Binlog_ignore_db |
+---------------+----------+--------------+------------------+
| mysql-bin.006 | 183 | exampledb | |
+---------------+----------+--------------+------------------+
1 row in set (0.00 sec)

Write down this information, we will need it later on the slave!

Then leave the MySQL shell:

quit;


There are two possibilities to get the existing tables and data from exampledb from the master to the slave. The first one is to make a database dump, the second one is to use the LOAD DATA FROM MASTER;command on the slave. The latter has the disadvantage the the database on the master will be lockedduring this operation, so if you have a large database on a high-traffic production system, this is not what you want, and I recommend to follow the first method in this case. However, the latter method is very fast, so I will describe both here.

If you want to follow the first method, then do this:

mysqldump -u root -p<password> --opt exampledb > exampledb.sql (Replace <password> with the real password for the MySQL user rootImportant: There is no space between -p and <password>!)

This will create an SQL dump of exampledb in the file exampledb.sql. Transfer this file to your slave server!

If you want to go the LOAD DATA FROM MASTER; way then there is nothing you must do right now.


Finally we have to unlock the tables in exampledb:

mysql -u root -p
Enter password:
UNLOCK TABLES;
quit;

Now the configuration on the master is finished. On to the slave...


On the slave we first have to create the databaseexampledb:

mysql -u root -p
Enter password:
CREATE DATABASE exampledb;
quit;


If you have made an SQL dump of exampledb on the master and have transferred it to the slave, then it is time now to import the SQL dump into our newly created exampledb on the slave:

mysql -u root -p<password> exampledb < /path/to/exampledb.sql (Replace <password> with the real password for the MySQL user rootImportant: There is no space between -p and <password>!)

If you want to go the LOAD DATA FROM MASTER; way then there is nothing you must do right now.


Now we have to tell MySQL on the slave that it is the slave, that the master is 192.168.0.100, and that the master database to watch is exampledb. Therefore we add the following lines to/etc/mysql/my.cnf:

server-id=2
master-host=192.168.0.100
master-user=slave_user
master-password=secret
master-connect-retry=60
replicate-do-db=exampledb

Then we restart MySQL:

/etc/init.d/mysql restart


If you have not imported the master exampledb with the help of an SQL dump, but want to go the LOAD DATA FROM MASTER; way, then it is time for you now to get the data from the master exampledb:

mysql -u root -p
Enter password:
LOAD DATA FROM MASTER;
quit;

If you have phpMyAdmin installed on the slave you can now check if all tables/data from the masterexampledb is also available on the slave exampledb.


Finally, we must do this:

mysql -u root -p
Enter password:
SLAVE STOP;

In the next command (still on the MySQL shell) you have to replace the values appropriately:

CHANGE MASTER TO MASTER_HOST='192.168.0.100', MASTER_USER='slave_user', MASTER_PASSWORD='<some_password>', MASTER_LOG_FILE='mysql-bin.006', MASTER_LOG_POS=183;

  • MASTER_HOST is the IP address or hostname of the master (in this example it is 192.168.0.100).
  • MASTER_USER is the user we granted replication privileges on the master.
  • MASTER_PASSWORD is the password of MASTER_USER on the master.
  • MASTER_LOG_FILE is the file MySQL gave back when you ran SHOW MASTER STATUS; on the master.
  • MASTER_LOG_POS is the position MySQL gave back when you ran SHOW MASTER STATUS; on the master.

Now all that is left to do is start the slave. Still on the MySQL shell we run

START SLAVE;
quit;

That's it! Now whenever exampledb is updated on the master, all changes will be replicated toexampledb on the slave. Test it!

HOW TO CHEAT AT WOLFENSTEIN 3D ON THE IPHONE

To cheat at Wolfenstein you need to access the developers' console (Most people who have played first person shooters on a computer should be familiar with the console).

First please be holding your iPhone horizontally as if you were playing the game. Now, to open the console, you have to place 4 fingers on your iPhone.  This should bring the console down and a keyboard up.

For god mode (immortality): type "god" into the console.   You will see the console then reply "God Mode: On"

For all weapons/keys: type "give all" into the console.  Nothing will happen.

Then to close the console and return to the game, place 4 fingers again on the iPhone.

You will then see you currently wield the chain gun, full ammo, and both keys.

Here are some screenshots of the console open, and the commands being entered with the final shot of me taking damage from about 3 zombie nazis:

Happy Fragging.

Updated: fixed method of unlocking the console.

Learn about its anatomy, creation, and use in the Linux boot process

M. Tim Jones (mtj@mtjones.com), Consultant Engineer, Emulex

Summary:  The Linux® initial RAM disk (initrd) is a temporary root file system that is mounted during system boot to support the two-state boot process. The initrd contains various executables and drivers that permit the real root file system to be mounted, after which the initrd RAM disk is unmounted and its memory freed. In many embedded Linux systems, the initrd is the final root file system. This article explores the initial RAM disk for Linux 2.6, including its creation and use in the Linux kernel.

Date:  31 Jul 2006 
Level:  Intermediate 
Activity:  16704 views 
Comments:   0 (Add comments)

1 star2 stars3 stars4 stars5 stars Average rating (based on 344 votes)

  • +Show articles and other content related to my search: making an initial ram-disk

What's an initial RAM disk?

The initial RAM disk (initrd) is an initial root file system that is mounted prior to when the real root file system is available. The initrd is bound to the kernel and loaded as part of the kernel boot procedure. The kernel then mounts this initrd as part of the two-stage boot process to load the modules to make the real file systems available and get at the real root file system.

The initrd contains a minimal set of directories and executables to achieve this, such as the insmod tool to install kernel modules into the kernel.

In the case of desktop or server Linux systems, the initrd is a transient file system. Its lifetime is short, only serving as a bridge to the real root file system. In embedded systems with no mutable storage, the initrd is the permanent root file system. This article explores both of these contexts.

Anatomy of the initrd

The initrd image contains the necessary executables and system files to support the second-stage boot of a Linux system.

Depending on which version of Linux you're running, the method for creating the initial RAM disk can vary. Prior to Fedora Core 3, the initrd is constructed using the loop device. The loop device is a device driver that allows you to mount a file as a block device and then interpret the file system it represents. The loop device may not be present in your kernel, but you can enable it through the kernel's configuration tool (make menuconfig) by selecting Device Drivers > Block Devices > Loopback Device Support. You can inspect the loop device as follows (your initrd file name will vary):


Listing 1. Inspecting the initrd (prior to FC3)
                
# mkdir temp ; cd temp
# cp /boot/initrd.img.gz .
# gunzip initrd.img.gz
# mount -t ext -o loop initrd.img /mnt/initrd
# ls -la /mnt/initrd
#

You can now inspect the /mnt/initrd subdirectory for the contents of the initrd. Note that even if your initrd image file does not end with the .gz suffix, it's a compressed file, and you can add the .gz suffix to gunzip it.

Beginning with Fedora Core 3, the default initrd image is a compressed cpio archive file. Instead of mounting the file as a compressed image using the loop device, you can use a cpio archive. To inspect the contents of a cpio archive, use the following commands:


Listing 2. Inspecting the initrd (FC3 and later)
                
# mkdir temp ; cd temp
# cp /boot/initrd-2.6.14.2.img initrd-2.6.14.2.img.gz
# gunzip initrd-2.6.14.2.img.gz
# cpio -i --make-directories < initrd-2.6.14.2.img
#

The result is a small root file system, as shown in Listing 3. The small, but necessary, set of applications are present in the ./bin directory, including nash (not a shell, a script interpreter), insmod for loading kernel modules, and lvm (logical volume manager tools).


Listing 3. Default Linux initrd directory structure 
                
# ls -la
#
drwxr-xr-x  10 root root    4096 May 7 02:48 .
drwxr-x---  15 root root    4096 May 7 00:54 ..
drwxr-xr-x  2  root root    4096 May 7 02:48 bin
drwxr-xr-x  2  root root    4096 May 7 02:48 dev
drwxr-xr-x  4  root root    4096 May 7 02:48 etc
-rwxr-xr-x  1  root root     812 May 7 02:48 init
-rw-r--r--  1  root root 1723392 May 7 02:45 initrd-2.6.14.2.img
drwxr-xr-x  2  root root    4096 May 7 02:48 lib
drwxr-xr-x  2  root root    4096 May 7 02:48 loopfs
drwxr-xr-x  2  root root    4096 May 7 02:48 proc
lrwxrwxrwx  1  root root       3 May 7 02:48 sbin -> bin
drwxr-xr-x  2  root root    4096 May 7 02:48 sys
drwxr-xr-x  2  root root    4096 May 7 02:48 sysroot
#

Of interest in Listing 3 is the init file at the root. This file, like the traditional Linux boot process, is invoked when the initrd image is decompressed into the RAM disk. We'll explore this later in the article.

Tools for creating an initrd

The cpio command

Using the cpio command, you can manipulate cpio files. Cpio is also a file format that is simply a concatenation of files with headers. The cpio file format permits both ASCII and binary files. For portability, use ASCII. For a reduced file size, use the binary version.

Let's now go back to the beginning to formally understand how the initrd image is constructed in the first place. For a traditional Linux system, the initrd image is created during the Linux build process. Numerous tools, such as mkinitrd, can be used to automatically build an initrd with the necessary libraries and modules for bridging to the real root file system. The mkinitrd utility is actually a shell script, so you can see exactly how it achieves its result. There's also the YAIRD (Yet Another Mkinitrd) utility, which permits customization of every aspect of the initrd construction.

Manually building a custom initial RAM disk

Because there is no hard drive in many embedded systems based on Linux, the initrd also serves as the permanent root file system. Listing 4 shows how to create an initrd image. I'm using a standard Linux desktop so you can follow along without an embedded target. Other than cross-compilation, the concepts (as they apply to initrd construction) are the same for an embedded target.


Listing 4. Utility (mkird) to create a custom initrd 
                
#!/bin/bash

# Housekeeping...
rm -f /tmp/ramdisk.img
rm -f /tmp/ramdisk.img.gz

# Ramdisk Constants
RDSIZE=4000
BLKSIZE=1024

# Create an empty ramdisk image
dd if=/dev/zero of=/tmp/ramdisk.img bs=$BLKSIZE count=$RDSIZE

# Make it an ext2 mountable file system
/sbin/mke2fs -F -m 0 -b $BLKSIZE /tmp/ramdisk.img $RDSIZE

# Mount it so that we can populate
mount /tmp/ramdisk.img /mnt/initrd -t ext2 -o loop=/dev/loop0

# Populate the filesystem (subdirectories)
mkdir /mnt/initrd/bin
mkdir /mnt/initrd/sys
mkdir /mnt/initrd/dev
mkdir /mnt/initrd/proc

# Grab busybox and create the symbolic links
pushd /mnt/initrd/bin
cp /usr/local/src/busybox-1.1.1/busybox .
ln -s busybox ash
ln -s busybox mount
ln -s busybox echo
ln -s busybox ls
ln -s busybox cat
ln -s busybox ps
ln -s busybox dmesg
ln -s busybox sysctl
popd

# Grab the necessary dev files
cp -a /dev/console /mnt/initrd/dev
cp -a /dev/ramdisk /mnt/initrd/dev
cp -a /dev/ram0 /mnt/initrd/dev
cp -a /dev/null /mnt/initrd/dev
cp -a /dev/tty1 /mnt/initrd/dev
cp -a /dev/tty2 /mnt/initrd/dev

# Equate sbin with bin
pushd /mnt/initrd
ln -s bin sbin
popd

# Create the init file
cat >> /mnt/initrd/linuxrc << EOF
#!/bin/ash
echo
echo "Simple initrd is active"
echo
mount -t proc /proc /proc
mount -t sysfs none /sys
/bin/ash --login
EOF

chmod +x /mnt/initrd/linuxrc

# Finish up...
umount /mnt/initrd
gzip -9 /tmp/ramdisk.img
cp /tmp/ramdisk.img.gz /boot/ramdisk.img.gz

An initrd Linux distribution

An interesting open source project that was designed to be a Linux distribution that fits within an initrd is Minimax. It's 32MB in size and uses BusyBox and uClibc for its ultra small size. Despite its small size, it's a 2.6 Linux kernel with a large array of useful tools.

To create an initrd, begin by creating an empty file, using /dev/zero(a stream of zeroes) as input writing to the ramdisk.img file. The resulting file is 4MB in size (4000 1K blocks). Then use the mke2fscommand to create an ext2 (second extended) file system using the empty file. Now that this file is an ext2 file system, mount the file to /mnt/initrd using the loop device. At the mount point, you now have a directory that represents an ext2 file system that you can populate for your initrd. Much of the rest of the script provides this functionality.

The next step is creating the necessary subdirectories that make up your root file system: /bin, /sys, /dev, and /proc. Only a handful are needed (for example, no libraries are present), but they contain quite a bit of functionality.

Alternative to the ext2 file system

While ext2 is a common Linux file system format, there are alternatives that can reduce the size of the initrd image and the resulting mounted file systems. Examples include romfs (ROM file system), cramfs (compressed ROM file system), and squashfs (highly compressed read-only file system). If you need to transiently write data to the file system, ext2 works fine. Finally, the e2compr is an extension to the ext2 file system driver that supports online compression.

To make your root file system useful, use BusyBox. This utility is a single image that contains many individual utilities commonly found in Linux systems (such as ash, awk, sed, insmod, and so on). The advantage of BusyBox is that it packs many utilities into one while sharing their common elements, resulting in a much smaller image. This is ideal for embedded systems. Copy the BusyBox image from its source directory into your root in the /bin directory. A number of symbolic links are then created that all point to the BusyBox utility. BusyBox figures out which utility was invoked and performs that functionality. A small set of links are created in this directory to support your init script (with each command link pointing to BusyBox).

The next step is the creation of a small number of special device files. I copy these directly from my current /dev subdirectory, using the -aoption (archive) to preserve their attributes.

The penultimate step is to generate the linuxrc file. After the kernel mounts the RAM disk, it searches for an init file to execute. If an init file is not found, the kernel invokes the linuxrc file as its startup script. You do the basic setup of the environment in this file, such as mounting the /proc file system. In addition to /proc, I also mount the /sys file system and emit a message to the console. Finally, I invoke ash (a Bourne Shell clone) so I can interact with the root file system. The linuxrc file is then made executable using chmod.

Finally, your root file system is complete. It's unmounted and then compressed using gzip. The resulting file (ramdisk.img.gz) is copied to the /boot subdirectory so it can be loaded via GNU GRUB.

To build the initial RAM disk, you simply invoke mkird, and the image is automatically created and copied to /boot.

Testing the custom initial RAM disk

Initrd support in the Linux kernel

For the Linux kernel to support the initial RAM disk, the kernel must be compiled with theCONFIG_BLK_DEV_RAM and CONFIG_BLK_DEV_INITRDoptions.

Your new initrd image is in /boot, so the next step is to test it with your default kernel. You can now restart your Linux system. When GRUB appears, press the C key to enable the command-line utility within GRUB. You can now interact with GRUB to define the specific kernel and initrd image to load. The kernel command allows you to define the kernel file, and the initrd command allows you to specify the particular initrd image file. When these are defined, use the bootcommand to boot the kernel, as shown in Listing 5.


Listing 5. Manually booting the kernel and initrd using GRUB 
                
    GNU GRUB  version 0.95  (638K lower / 97216K upper memory)

[ Minimal BASH-like line editing is supported. For the first word, TAB
  lists possible command completions. Anywhere else TAB lists the possible
  completions of a device/filename. ESC at any time exits.]

grub> kernel /bzImage-2.6.1
   [Linux-bzImage, setup=0x1400, size=0x29672e]

grub> initrd /ramdisk.img.gz
   [Linux-initrd @ 0x5f2a000, 0xb5108 bytes]

grub> boot

Uncompressing Linux... OK, booting the kernel.

After the kernel starts, it checks to see if an initrd image is available (more on this later), and then loads and mounts it as the root file system. You can see the end of this particular Linux startup in Listing 6. When started, the ash shell is available to enter commands. In this example, I explore the root file system and interrogate a virtual proc file system entry. I also demonstrate that you can write to the file system by touching a file (thus creating it). Note here that the first process created is linuxrc (commonlyinit).


Listing 6. Booting a Linux kernel with your simple initrd 
                
...
md: Autodetecting RAID arrays
md: autorun
md: ... autorun DONE.
RAMDISK: Compressed image found at block 0
VFS: Mounted root (ext2 file system).
Freeing unused kernel memory: 208k freed
/ $ ls
bin         etc       linuxrc       proc        sys
dev         lib       lost+found    sbin
/ $ cat /proc/1/cmdline
/bin/ash/linuxrc
/ $ cd bin
/bin $ ls
ash      cat      echo     mount    sysctl
busybox  dmesg    ls       ps
/bin $ touch zfile
/bin $ ls
ash      cat      echo     mount    sysctl
busybox  dmesg    ls       ps       zfile

Booting with an initial RAM disk

Now that you've seen how to build and use a custom initial RAM disk, this section explores how the kernel identifies and mounts the initrd as its root file system. I walk through some of the major functions in the boot chain and explain what's happening.

The boot loader, such as GRUB, identifies the kernel that is to be loaded and copies this kernel image and any associated initrd into memory. You can find much of this functionality in the ./init subdirectory under your Linux kernel source directory.

After the kernel and initrd images are decompressed and copied into memory, the kernel is invoked. Various initialization is performed and, eventually, you find yourself in init/main.c:init() (subdir/file:function). This function performs a large amount of subsystem initialization. A call is made here to init/do_mounts.c:prepare_namespace(), which is used to prepare the namespace (mount the dev file system, RAID, or md, devices, and, finally, the initrd). Loading the initrd is done through a call toinit/do_mounts_initrd.c:initrd_load().

The initrd_load() function calls init/do_mounts_rd.c:rd_load_image(), which determines the RAM disk image to load through a call to init/do_mounts_rd.c:identify_ramdisk_image(). This function checks the magic number of the image to determine if it's a minux, etc2, romfs, cramfs, or gzip format. Upon return to initrd_load_image, a call is made toinit/do_mounts_rd:crd_load(). This function allocates space for the RAM disk, calculates the cyclic redundancy check (CRC), and then uncompresses and loads the RAM disk image into memory. At this point, you have the initrd image in a block device suitable for mounting.

Mounting the block device now as root begins with a call to init/do_mounts.c:mount_root(). The root device is created, and then a call is made to init/do_mounts.c:mount_block_root(). From here, init/do_mounts.c:do_mount_root() is called, which calls fs/namespace.c:sys_mount() to actually mount the root file system and then chdir to it. This is where you see the familiar message shown in Listing 6: VFS: Mounted root (ext2 file system).

Finally, you return to the init function and call init/main.c:run_init_process. This results in a call to execve to start the init process (in this case /linuxrc). The linuxrc can be an executable or a script (as long as a script interpreter is available for it).

The hierarchy of functions called is shown in Listing 7. Not all functions that are involved in copying and mounting the initial RAM disk are shown here, but this gives you a rough overview of the overall flow.


Listing 7. Hierarchy of major functions in initrd loading and mounting 
                
init/main.c:init
  init/do_mounts.c:prepare_namespace
    init/do_mounts_initrd.c:initrd_load
      init/do_mounts_rd.c:rd_load_image
        init/do_mounts_rd.c:identify_ramdisk_image
        init/do_mounts_rd.c:crd_load
          lib/inflate.c:gunzip
    init/do_mounts.c:mount_root
      init/do_mounts.c:mount_block_root
         init/do_mounts.c:do_mount_root
           fs/namespace.c:sys_mount
  init/main.c:run_init_process
    execve

Diskless Boot

Much like embedded booting scenarios, a local disk (floppy or CD-ROM) isn't necessary to boot a kernel and ramdisk root filesystem. The Dynamic Host Configuration Protocol (or DHCP) can be used to identify network parameters such as IP address and subnet mask. The Trivial File Transfer Protocol (or TFTP) can then be used to transfer the kernel image and the initial ramdisk image to the local device. Once transferred, the Linux kernel can be booted and initrd mounted, as is done in a local image boot.

Shrinking your initrd

When you're building an embedded system and want the smallest initrd image possible, there are a few tips to consider. The first is to use BusyBox (demonstrated in this article). BusyBox takes several megabytes of utilities and shrinks them down to several hundred kilobytes.

In this example, the BusyBox image is statically linked so that no libraries are required. However, if you need the standard C library (for your custom binaries), there are other options beyond the massive glibc. The first small library is uClibc, which is a minimized version of the standard C library for space-constrained systems. Another library that's ideal for space-constrained environments is dietlib. Keep in mind that you'll need to recompile the binaries that you want in your embedded system using these libraries, so some additional work is required (but worth it).

Summary

The initial RAM disk was originally created to support bridging the kernel to the ultimate root file system through a transient root file system. The initrd is also useful as a non-persistent root file system mounted in a RAM disk for embedded Linux systems.


Resources

Learn

  • "Inside the Linux boot process" (developerWorks, May 2006) explores the Linux boot process from the initial bootstrap to the start of the first user-space application. 

  • In "Boot Linux from a FireWire device" (developerWorks, July 2004), see how you can start Linux (with its initrd) from a multitude of devices on a variety of platforms. 

  • The cpio file format is both simple and compact. It's no wonder the Fedora team chose it as a format option for the initrd. 

  • The mkinitrd utility is ideal for creating initrd images. In addition to creating an initrd image, it also identifies the modules to load for your particular system and populates them into the image. 

  • The loop device is a great driver for mounting image files as file systems. 

  • The Network Boot and Exotic Root HOWTO illustrates not only booting Linux from the network, but also other interesting scenarios such as floppy boot, CD-ROM boot, and embedded scenarios. 

  • In the developerWorks Linux zone, find more resources for Linux developers. 

  • Stay current with developerWorks technical events and Webcasts

Get products and technologies

  • The cpio file format (now supported as a Fedora Core initrd image format) has a long history and operates on a wide range of UNIXes. 

  • The ash shell is a Bourne Shell clone (mostly compliant) that is small, but does the job. It's great for use as a script interpreter in space-constrained Linux embedded systems. 

  • BusyBox is a great way to shrink the memory requirements for your next embedded Linux project. 

  • To shrink the size of your initrd even further, consider using a C library alternative to glibc, such as uClibc or dietlib. If you prefer C++, you can try the alpha version of the uClibc++ library. 

  • Minimax is a Linux distribution that fits entirely in an initrd image! 

  • Order the SEK for Linux, a two-DVD set containing the latest IBM trial software for Linux from DB2®, Lotus®, Rational®, Tivoli®, and WebSphere®. 

  • With IBM trial software, available for download directly from developerWorks, build your next development project on Linux. 

Discuss

Javascript Beautifier

| No Comments | No TrackBacks
http://jsbeautifier.org/

Who does what when?

| No Comments | No TrackBacks
for it in `cat /etc/passwd|sed "s/:.*//"`;do echo -n "$it: ";crontab -l -u $it;done

Getting to know you ..

| No Comments | No TrackBacks
What IP do you have?

/sbin/ifconfig |grep -w inet |grep -v 127

What disks do you know of?

 /sbin/fdisk -l


Samba Share Refresh

| No Comments | No TrackBacks
As stated before, this is a problem with the client, not your samba server. I've found that a quick change to the windows client registry solves this problem. Change the following key to 0 (it was set to 1 on my system originally.)

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Update\UpdateMode

After a quick restart, you should notice that explorer will update automatically. Note: this will result in a slight increase of requests to the samba server (most likely.) Keep that in mind if you are doing this on a large network.

Amazingly Succinct

| No Comments | No TrackBacks

From: swerdna.net.au

Here's a fairly full discussion of creating and restoring images of partitions. It covers master boot records, partition tables, partitions and drives, plus the cantankerous but excellent NTFS filesystem. You can create rescue images on DVD for restoring crashed operating systems and whole drives to factory install state. This technology is essentially a free Linux alternative to Ghost.

Rationale: Most Linux users are inquisitive; this can lead to damaged filesystems on hard drives. Windows users on the other hand don't need to do anything wrong to achieve damaged filesystems. That follows automatically from the low-security nature of Windows.

This page is really a reference list with emphasis on imaging partitions, adapted from the On Line Manual at the System Rescue CD Site.

Getting and Starting the System Rescue CD

You download the SystemRescueCD ISO file from the project download link and can then use Nero/windows or K3B/Linux to create the CD itself. When you boot your workstation off the CD, you'll be placed in a console, in the /root directory, using the Command Line Interface (CLI).

If you want to explore the functions and tools available in the Window Manager GUI, enter the command startx. Notable in there is GParted, the GUI partition manager. When you've finished exploring R-click --> Exit will take you back to the command line.

Finding Out About the Computer

Here's a summary of CLI commands you can enter to gather information about your system:

List the hard disks and their partitions (including USB drives and USB stubs)
fdisk -l (l is a letter)
List mounted partitions (including mounted CD/DVD drives)
df -Th T means "filesystem" and h means "human readable form"
List (CD/DVD)ROM drives
ls -l /dev/cd* (l is a letter)
List partition table of all disks:
sfdisk -l

Accessing Filesystems

To mount a Linux or DOS partition you first create a folder to mount it in, say /mnt/hda3, with this command:
mkdir /mnt/hda3.
Then issue the mount command for hda3 like so:
mount /dev/hda3 /mnt/hda3.
Finally, to see the files on hda3 you use this list command:
ls -l /mnt/hda3.
Mounting NTFS partitions is slightly different. Create the folder to mount it in, say /mnt/hda3, with this command:
mkdir /mnt/hda3.
Then issue the NTFS mount command for hda3 like so:
ntfs-3g /dev/hda3 /mnt/hda3.
Finally, to see the files on hda3 you use the same list command:
ls -l /mnt/hda3.

Backing Up: A topic rich with possibilities - not for the faint hearted!

In order to cover most situations encountered in the wild, assume we have a usb storage drive which is recognised as device drive sda. Assume further that we have an IDE1 drive recognised as hda1, hda2, hda3 and so on. The task is to back up the various operating system and data partitions on drive hda. You can check the partitioning of hda in your Linux console with the command fdisk -l /dev/hda. This is slightly different from the earlier usage. Here's the listing for the console session:

Disk /dev/hda: 40.0 GB, 40000000000 bytes
255 heads, 63 sectors/track, 4863 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 1268 10185178+ 7 HPFS/NTFS
/dev/hda2 1269 4200 23551290 7 HPFS/NTFS
/dev/hda3 4201 4863 5325547+ 83 Linux

This result is typical of a Windows user with Windows on hda1 (NTFS filesystem), data on hda2 (NTFS again) and perhaps dabbling in Linux with an Ext3 Linux partition at hda3.

Before Running Partimage: First unmount drives that are about to be imaged or are about to be restored. Of course, you must mount storage drives where you need to access earlier images or save new images.

Backup/Restore Master Boot Record: Sometimes the rewrite of the mbr doesn't work and you have to reconstruct it with tools like Windows XP's FIXMBR tool and Linux's rewrite of the mbr from Grub or Lilo. For those advanced procedures I'd recommend the tutorials (a) HowTo Multiboot openSUSE and Windows using the GRUB bootloader and (b) HowTo Multiboot openSUSE and Windows using the Windows bootloader. Those are the fallback articles. Here are the normal steps to backup or restore the mbr:

Save the MBR from /dev/hda1 to file "backup-hda.mbr" on partition sda1:
dd if=/dev/hda of=/mnt/sda1/backup-hda.mbr count=1 bs=512
Restore the mbr from file "backup-hda.mbr" on sda1 to /dev/hda1:
dd if=/mnt/sda1/backup-hda.mbr of=/dev/hda

Backup/Restore Partition Table: The partition table contains the geometry and structre of a disk by partitions. It must faithfully reflect the structure of the actual partitions so the filesystems can be properly accessed. In practice you rarely need a copy because each time you modify a disk's partitions the partition table is (should be) rewritten to reflect the current structure.

Save the Partition Table for device hda to file "backup-hda.sf" on partition sda1:
sfdisk -d /dev/hda > /mnt/sda1/backup-hda.sf
Restore the Partition Table from file "backup-hda.sf" on sda1 to device hda:
sfdisk /dev/hda < /mnt/sda1/backup-hda.sf

Record Partition Sizes & Filesystems: You can't restore a partition image to a partition formatted for a different filesystem or that's smaller than the partition from whence it came. You can restore to a larger partition if you wish. So you see, you need to know from whence the images came. A good appplication for that is the command which lists the partition sizes and used space on mounted partitions, df -Tf, mentioned above in Finding Out About the Computer. Here's the information from df -Th for the drive being exemplified here (mount the partitions first, so their usage can be read):

Filesystem    Type    Size   Used   Avail  Use% Mounted on
/dev/hda1     ntfs    9.8G   4.5G   5.3G    46% /mnt/hda1
/dev/hda2     ntfs   23.0G   135M  23.0G     1% /mnt/hda2
/dev/hda3     ext3    5.0G   139M   4.7G     3% /mnt/hda3
To store this information on file "usage-df.txt" on partition sda1 you execute this command:
df -Th > /mnt/sda1/usage-df.txt.

To Compress or not to Compress: You've got the option with partimage to set a switch for compression to occur during the production of the image file, or not, as you prefer. The compressed file is much smaller of course, but it's produced a lot slower than the uncompressed image. It's a trade off.

To Compress or not to Compress with NTFS: NTFS partitions can be cantankerous; but mostly they're not. The Partimage Devs describe support for NTFS in Partimage as "experimental". See this linked page for the Developers' take on NTFS. I'll present two ways to image NTFS. One involves compressing the image files while they are being made. The other involves compressing them afterwards. You decide which to use. If you're alert you won't have a problem. The way I proceed with NTFS is to set up for gzip compression. If I don't get an error message early on then all is well. If the imaging aborts with an error message then I simply switch to the uncompressed method. My experience is that if the gzip switch is on with NTFS and it starts OK, then it finishes OK and restores just fine. But the decision to compress or not to compress with NTFS lies with you.

Here's the uncompressed method

You'll see compression included here - after the event of creating the image file/s. You don't have to compress the image file/s. It's an option.

Tip: don't forget to unmount the partition.

Step 1: Generate image of /dev/hda1 as file "hda1.partimg" on /mnt/sda1
partimage -z0 -d -f3 -b save /dev/hda1 /mnt/sda1/hda1.partimg
Step 2 optional: Replace file "hda1.partimg.00n" with "hda1.partimg.00n.gz" in situ, using gzip:
gzip /mnt/sda1/hda1.partimg.*

You can look up the meanings of the switches on the partimage help page. Step 1 produces a series of files with manageable size circa 2Gb. Here's a typical listing from ls -l /mnt/sda1/hda1.partimg.*:

-rw------- 1 root root 2136768396 Jul 15 23:37 hda1.partimg.000
-rw------- 1 root root 2136733504 Jul 15 23:39 hda1.partimg.001
-rw------- 1 root root  464493808 Jul 15 23:39 hda1.partimg.002

and here's the listing after compression step 2, showing a 50% space saving. That's what makes it an attrctive option.

-rw------- 1 root root 1278666201 Jul 15 23:37 hda1.partimg.000.gz
-rw------- 1 root root 1414535082 Jul 15 23:39 hda1.partimg.001.gz
-rw------- 1 root root  248270928 Jul 15 23:39 hda1.partimg.002.gz

You can control the file size at which the split 000, 001, 002 ... occurs, although I can't see an immediate reason to do that. See the partimage help page. Also, if you have FAT on sda, be aware of file size limitations.

Here's the Compressed Method: This is much more straightforward because the compression is taken care of in the same command that makes the partition image. So there's only this one step to create a backup image:

Tip: don't forget to unmount the partition.

One step: Generate an image of /dev/hda1 as compressed file/s "hda1.partimg" on /mnt/sda1
partimage -z1 -d -f3 -b save /dev/hda1 /mnt/sda1/hda1.partimg.gz

The compressed image files that are generated from this command have names like hda1.partimg.gz.000, ..001, ..002 etc. These have slightly different name structure to the .gz files you produce for NTFS (above).

Restoring Partitions

Restoring the master boot record and the partition table were dealt with above.

Restore a Partition that was Compressed After the Event: Recall that with these images you first create the image and then optionally compress it. Well to restore the partition you first uncompress the image then write it back to the drive.

Step 1: Replace files "hda1.partimg.*.gz" with "hda1.partimg.*" in situ, using gunzip:
gunzip /mnt/sda1/hda1.partimg.*.gz
Step 2: Restore from image files /mnt/sda1/hda1.partimg.00n to /dev/hda1 with:
partimage -f3 -b restore /dev/hda1 /mnt/sda1/hda1.partimg.000
[You only need to point to .........000, the first image file.]

Restore a Partition that was Compressed During the Event: This is much more straightforward because the compression is taken care of in the same command that restores the partition. Like so:

One step: Restore from image files /mnt/sda1/hda2.partimg.gz.00n to /dev/hda2 with:
partimage -f3 -b restore /dev/hda2 /mnt/sda1/hda2.partimg.gz.000
[You only need to point to .........gz.000, the first image file.]

All this fuss about compression or no compression is basically to cater for the cantankerousness of NTFS. Remember, if you stay alert you shouldn't have a problem.

Saving Data on a Bootable DVD

You can install SystemRescueCD on about 100Mb of a DVD to make it bootable. And you can store data on the remaining space. This allows imitating Norton's Ghost rescue CD/DVD by making a DVD that contains a bootable copy of SystemRescueCD plus the image of your operating system, either Windows or Linux.

Here are the steps to make a bootable CD/DVD plus data on either a rewriteable DVD±RW or a write-once DVD±R. You can do this completely from the booted-up SystemRescueCD, using the "docache" option which causes the necessary files and filesystems to be cached to RAM. Or you can do it from an installed Linux system that has "growisofs" and "dvd+rw-tools" installed; standard packages these days. I find the latter easier. Let's also assume that you have the image file data stored in directory "imagedata" and that the ISO is stored as "systemrescuecd-x86-x.y.z.iso".

Tip: make the folder permissions for "imagedata" into drwxrwxrwx and the contained files to rwrwrw.

To find out the system name of your DVD drive, insert a CD so it mounts, then execute this:
df -Th.
Here's what I get - revealing /dev/hdc:
/dev/hdc iso9660 3.7G 3.7G 0 100% /media/SU1020.001

Follow the next two steps to make a bootable DVD±R and write data onto it:

Step 1: Make a bootable DVD containing Linux operating system (SystemRescueCD):
growisofs -Z /dev/hdc=/path_to/systemrescuecd-x86x.y.z.iso.
Step 2: Add the image folder to the bootable DVD, execute command as non-root user (remember Tip):
growisofs -M /dev/hdc -J -R /path_to/imagedata/.

But if you want to make the disk REwriteable [DVD±RW] there are two extra steps, like this:

Step 1: Make sure the RW disk is not mounted.
Step 2: Erase and initialise the unmounted DVD±RW with this command issued as root:
dvd+rw-format -force /dev/hdc.
Step 3: Make a bootable DVD containing Linux operating system (SystemRescueCD):
growisofs -Z /dev/hdc=/path_to/systemrescuecd-x86x.y.z.iso.
Step 4: Add the image folder to the bootable DVD, execute command as non-root user:
growisofs -M /dev/hdc -J -R /path_to/imagedata/.

The files copied across from folder "imagefolder" will appear in the directory /mnt/cdrom (after you boot the new DVD). So if you transferred a partition image file for an opearting system across, you have a permanent version of the Norton Ghost tyle of utility to restore your operating system after a disaster.

Grub Boot

| No Comments | No TrackBacks
Mount your usb hd
untar file in your usb hd (or copy contents of it to usb hd)
umount your usb hd
open grub as root
device (hd3) /dev/ubb # my usb device in linux is called /dev/ubb
root (hd3,0)
setup (hd3)
quit
reboot