Tuesday, February 15, 2005

Adding web counter to blog

Easiest way to do this is to use a counter available at:

sitemeter.com

It even has an option after you've signed in to automagically insert the html code to carry out the web counting - and it's FREE!

Backing up a harddisk in Linux

I want to create lots of DOS machines with old 386's and 486's and I don't see myself sitting with old stiffies and floppies installing DOS on each machine. So I decided to go for the option of installing the harddrive of the DOS machine I want to create and taking an image file of the original singing dancing games, worprocessing, killer DOS machine an reimaging this second harddrive from this image.

Here are some good links of Free Linux tools

Free Hard Disk and Partition Imaging and Backup Software

backing up entire hard disks or partitions using dd

Looks like partimage will be the way to go

Sunday, February 06, 2005

Config files that are called when Linux boots

This has always confused the peanut butter out of me

OK so first it runs all the BIOS stuff, loads the Master Boot Record (MBR) which looks for a bootable partition and then the fun starts.

If you have LILO or GRUB, it will load up the boot manager which in turn will load a kernel based on the option in the menu. For example you can have two kernel versions in /boot. One called /boot/vmlinux.orig and on called /boot/vmlinux.new. Based on which menu option you choose you can get LILO to load up either two of these.

Once your system boots, you will see the kernel being loaded and started. As it is loaded and begins to execute, you will see screens of information flash past - which is usually information about drivers that are being loaded into memory. To check what happened when the kernel booted espeically if you want to debug a driver which didn;t load - you can run dmesg or look at /var/log./messages.

Next the kernel starts this init process which reads the /etc/inittab file to decide what to do. The dirst thing init runs out of the inittab is the script /etc/rc.d/rc.sysinit

As with everything else under /etc/rc.d, this is a shell script, so you can take a look at it if you want. Actually, I feel that looking through and becoming familiar with which scripts does what and it what order is a good way of learning about your system.

Among the myriad of things done here are checking and mounting file systems, removing old lock and PID files, and enabling the swap space.

Note that if the file system check notes some serious problems, the rc.sysinit will stop and bring you to a shell prompt, where you can attempt to clean up by hand. Once you exit this shell, the next command to be executed (aside from an echo) is a reboot. This is done to ensure the validity of the file systems.

Next, init looks through inittab for the line with initdefault in the third field. The initdefault entry tells the system what run-level to enter initially, normally run-level 3 (without X Windows) or run-level 5 (with X Windows). Other systems have the default run-level 1 to bring you into single-user or maintenance mode. Here you can perform certain actions without worrying users or too many other things happening on your system. (Note: You can keep users out simply by creating the file /etc/nologin. See the nologin man-page for details.)

What kind of actions can you perform here? The action with the most impact is adding new or updating software. Often, new software will affect old software in such a way that it is better not to have other users on the system. In such cases, the installation procedures for that software should keep you from installing unless you are in maintenance mode.

This is also a good place to configure hardware that you added or otherwise change the kernel. Although these actions rarely impact users, you will have to do a kernel rebuild. This takes up a lot of system resources and degrades overall performance. Plus, you need to reboot after doing a kernel rebuild and it takes longer to reboot from run-level 3 than from run-level 1.

If the changes you made do not require you to rebuild the kernel (say, adding new software), you can go directly from single-user to multi-user mode by running init 3. The argument to init is simply the run level you want to go into, which, for most purposes, is run-level 3. However, to shut down the system, you could bring the system to run-level 0 or 6. (See the init man-page for more details.)

Init looks for any entry that has a 3 in the second field. This 3 corresponds to the run-level where we currently are. Run-level 3 is the same as multi-user mode.

Within the inittab, there is a line for every run level that starts the script /etc/rc.d/rc, passing the run level as an argument. The /etc/rc.d/rc script, after a little housekeeping, then starts the scripts for that run level. For each run level, there is a directory underneath /etc/rc.d, such as rc3.d, which contains the scripts that will be run for that run level.

In these directories, you may find two sets of scripts. The scripts beginning with K are the kill scripts, which are used to shutdown/stop a particular subsystem. The S scripts are the start scripts. Note that the kill and start scripts are links to the files in /etc/rc.d/init.d. If there are K and S scripts with the same number, these are both linked to the same file.

This is done because the scripts are started with an argument of either start or stop. The script itself then changes its behavior based on whether you told it to start or stop. Naming them something (slightly) different allows us to start only the K scripts if we want to stop things and only the S scripts when we want to start things.

When the system changes to a particular run level, the first scripts that are started are the K scripts. This stops any of the processes that should not be running in that level. Next, the S scripts are run to start the processes that should be running.

Let's look at an example. On most systems, run-level 3 is almost the same as run-level 2. The only difference is that in run-level 2, NFS is not running. If you were to change from run-level 3 to run-level 2, NFS would go down. In run-level 1 (maintenance mode), almost everything is stopped.


Most users are only familiar with two run states or run levels. The one that is most commonly experienced is what is referred to a multiuser mode. This is where logins are enabled on terminals when the network is running and the system is behaving "normally." The other run level is system maintenance or single-user mode, when only a single user is on the system (root), probably doing some kind of maintenance tasks. Although it could be configured to allow logins by other users, usually the system is so configured that only one login is allowed on the system console.

On every system that I have encountered, Linux will automatically boot into run-level 3. This is the normal operating mode. To get to a lower run level (for example, to do system maintenance), the system administrator must switch levels manually.

It is generally said that the "system" is in a particular run-level. However, it is more accurate to say that the init process is in a particular run level, because init determines what other processes are started at each run-level.

In addition to the run levels most of us are familiar with, there are several others that the system can run in. Despite this fact, few of them are hardly ever used. For more details on what these run levels are, take a look at the init man-page.

The system administrator can change to a particular run level by using that run level as the argument to init. For example, running init 2 would change the system to run-level 2. To determine what processes to start in each run level, init reads the /etc/inittab file. This is defined by the second field in the /etc/inittab file. Init reads this file and executes each program defined for that run level in order. When the system boots, it decides what run level to go into based on the initdefault entry in /etc/inittab.

The fields in the inittab file are:

idunique identity for that entry
rstate run level in which the entry will be processed
action tells init how to treat the process specifically
process what process will be started

One thing I need to point out is that the entries in inittab are not run exactly according to the order in which they appear. If you are entering a run level other than S for the first time since boot-up, init will first execute those entries with a boot or bootwait in the third column. These are those processes that should be started before users are allowed access to the system, such as checking then mounting the status of the file systems.

In run-level 3, the /sbin/mingetty process is started on the terminals specified. The getty process gives you your login: prompt. When you have entered your logname for the first time, getty starts the login process, which asks you for your password. If your password is incorrect, you are prompted to input your logname again. If your password is correct, then the system starts your "login shell. " Note that what gets started may not be a shell at all, but some other program. The term "login shell" is the generic term for whatever program is started when you login. This is defined by the last field of the corresponding entry in /etc/passwd.

Keep in mind that you can move in either direction, that is, from a lower to higher run level or from a higher to lower run level without having to first reboot. init will read the inittab and start or stop the necessary processes. If a particular process is not defined at a particular run level, then init will kill it. For example, assume you are in run-level 3 and switch to run-level 1. Many of the processes defined do not have a 1 in the second field. Therefore, when you switch to run-level 1, those processes and all their children will be stopped.

If we look at the scripts in rc1.d, we see there all the scripts are kill scripts, with the exception of one start script. It is this start script that actually kills all the processes. It does exec init -t1 S, which brings the system into maintenance mode in one (-t1) minute.

To shutdown the system immediately, you could run

which will bring the system immediately into run-level 0. As with run-level 1, there is only one start script for run-level 0. It is this script that kills all the processes, unmounts all the file systems, turns off swap, and brings the system down.

After it has started the necessary process from inittab, init just waits. When one of its "descendants" dies (a child process of a child process of a child process, etc., of a process that init started), init rereads the inittab to see what should be done. If, for example, there is a respawn entry in the third field, init will start the specified process again. This is why when you log out, you immediately get a new login: prompt.

Because init just waits for processes to die, you cannot simply add an entry to inittab and expect the process to start up. You have to tell init to reread the inittab. However, you can force init to reread the inittab by running init (or telinit) Q.

In addition to the run levels we discussed here, several more are possible. There are three "pseudo" run-levels a, b, and c. These are used to start specific programs as needed or "on demand". Any listed in inittab with the approapriate run-level will be started, however no actual run-level change occurs. If you're curious about the details, take a look at the init(8) man-page or the section on init-scripts.

ActionMeaning
bootExecuted during system boot.
bootwaitExecuted during system boot, but init waits until they have completed.
initdefaultThe default run level init starts after the system boots.
ondemandExecuted when one of the "on demand" run levels is called (a,b, and c)
powerwaitExecuted when the system power fails. Init will wait until the command completes.
powerfailExecuted when the system power fails, but init will not wait for completion.
powerokwaitExecuted when init is informed that power has been restored.
powerfailnowExecuted when init is informed that the external battery is empty.
resumeExecuted when init is told by the kernel that "Software Suspend"
sysinitExecuted during system boot before any boot or bootwait entries.
respawnRestarted if the processes stops.
waitStarted once when the specific run-level is entered and init waits for completion.
onceStarted once when the specific run-level is entered but init does not wait for completion.
ctrlaltdelExecute when someone on the system console presses CTRL-ALT-DEL.

Table - List of inittab actions.

If necessary, you can add your own entries into /etc/inittab. However, what is typically done is that init-scripts are added to the appropriate directory for the run-level where you want to start it. Depending on your Linux distribution, you could simply copy it into /etc/rc.d and use the appropriate admin tool, like Yast2 to add the script to the appropriate directory. For more details see the section on init-scripts.

Note however, that simply changing /etc/inittab is not enough. You need to tell the init process to re-read it. Normally init will re-read the file when it changes run levels or by sending it a hangup signal with

kill -HUP .

Also running the command

telinit q

will tell init to reread it.

Be extremely careful if you edit the /etc/inittab file by hand. An editing mistake could prevent your system from booting into a specific run level. If you use the boot, bootwait or sysinit actions, you could prevent your system from booting at all. Therefore, like with any system file it is good idea to make a backup copy first. If you make a mistake that prevents a particular program from starting, and the action is respawned, init might get caught in a loop. That is, init tries to start the program, cannot, for whatever reason and then tries to start it again. If init finds that it is starting the program more than 10 times within 2 minutes, it will treat this as an error and stops trying. Typically you will get messages in the system log that the process is "respawning too rapidly".

If you were to just install the Linux operating system on your hard disk, you would not be able to do very much. What actually makes Linux so useful is all of the extra things which are brought with it. This is essentially true for every operating system.

What makes Linux so useful as well as powerful are all the of services, which are generally referred to as daemons. These daemons typically run without user intervention providing everything from printing to file services to Web pages and beyond. Because they are not part of the operating system proper they are normally loaded separately from the kernel. Although many of these services could be made part of the kernel, they are mostly separate programs. Because they are separate programs something needs to configure and start them.

In most cases, simply installing a particular package is sufficient to activate the appropriate daemon. However, there are times when you need to make changes to how these demons behave, which often means changing the way the program starts up. In order to be able to do that, you obviously need to know just how and where these daemons are started in the first place. That's exactly what we're going to talk about here.

Once the kernel is loaded, one of the last things it does is to start the init process. The job of the init process (or simply init) is to start all of the daemons at the appropriate time. What the appropriate time is depends on a number of different things. For example, you may be performing administrative tasks and you do not want certain daemons to be running. Although you can stop those demons you do not need, the system provides a mechanism to do this automatically.

To understand this mechanism we need to talk about something called "run states" or "run levels". Most users (and many administrators, for that matter) are familiar with only one run level. This is the run level in which the system is performing all of its normal functions. Users can login, submit print jobs, access Web pages, and do everything else one would expect. This run level is commonly referred to as multiuser mode. In contrast, maintenance or single user mode is normally recommended for administrative tasks.

Each run level is referred to by its number. When the system is not doing anything, that is the system is stopped, this is run level 0. Single user mode is run-level 1. Multiuser mode is actually multiple runs levels. Depending on which distribution or which version of Unix you ahve, this can be run-level 2, run-level 3 and run-level 5. Most Linux systems automatically booting into run-level 3 when the system starts. Run level 2 is very similar to run level 3, although a number of things do not run in level 2. In fact, on some systems (SCO UNIX for example), run level 2 is the standard multi-user mode. Run-level 5 is where the GUI starts automatically. (For more details on the run levels, take a look at the init(8) man-page.)

Like many other aspects of the system, init has its own configuration file: /etc/inittab (see the table below). This file contains the init table (inittab), which tells it init what to do and when to do it. Each activity init does is represented by a single line in the inittab, which consists of four entries, separated by a colon. The first field is a unique identifier for that entry, which enables init to keep track of each daemon as it runs. The second field is the run level in which each particular entry is run.

The third entry is the action, which tells init how to behave in regard to this entry. For example, some entries are only processed when the system boots. Others are automatically re-started should that particular process stop (such as terminal logins). The last entry is what program will be started and often a number of options for that program.

If you take look in inittab on your system you may notice something peculiar. More than likely, you are not going to find any entries for the system demons we have been talking about. The reason is quite simply that the daemons are not started through the inittab, but rather through scripts which are started from the inittab. These scripts we see as the entries labeled l0 through l6, for run levels 0 through 6 (the letter "ell", not the number one).

In the example below, the "action" is that init waits until the program has terminated before continuing on and processing other entries for this run level. This also means that the entry will only be processed once as the system enters that particular one level.

The key to all of this is the program which is run for each run level. In every case, it is the shell script rc, which is given the appropriate run level as an argument. This script is often called the "run level master script" as it is responsible for loading all of the other init scripts. Where this script lies and what it is called will be different for different Linux distributions. Under older versions of SuSe it in /etc/rc.d, but now it's in /etc/init.d/. Under Caldera the script resides under /etc/rc.d. Note that starting with version 8.0, SuSe also has an /etc/rc.d directory, which is actually a symbolic link to /sbin/init.d.

Not just the location of the script is different between distributions, but so is the actual code. However, the basic functionality is generally the same. That is, to start other scripts which finally start the daemons we have been talking about all along.

One of the key aspects is how the system determines which daemon to start in which run level. As you might guess, this is accomplished through the run-level that is passed as an argument to the RC script. At least that's part of it. In addition, the system needs a list of which scripts should be started in which run level. This is accomplished not by a text file, but rather by separating the programs or scripts into different directories, one for each run level.

If you look in the /sbin/init.d or /etc/rc.d directory you'll see a number of subdirectories of the form rc#.d, where # is a particular run level. For example, the directory rc3.d is for run level 3. Within the subdirectories are not the actual scripts, as you might have guessed, but rather symbolic links to the actual scripts. The primary reason for this is that a script can be started in more than one run level. If the files were not links, but rather copies, any change would have to be made to every copy. The reason they are symbolic links, is that they may point to files on other file systems which is only possible by using symbolic links.

With SuSe, the /sbin/init.d directory is also where the real scripts reside. On Caldera, the scripts reside under /etc/rc.d/init.d.

At first glance, the filenames may be a little confusing. Although it is fairly simple to figure out what daemon is started by looking at the name, the way these links are named takes a little explanation.

As you might guess, the link ending in "apache" points to the script which starts the Apache Web server.

However, you'll see there are two files with this ending. The really odd thing is that both of these links point to the exact same file. So, what's the deal?

Part of the explanation lies in the first letter of each of these links. As you see, each starts with either the letter S or the letter K. Those which begin with the letter S are used to start the particular service and those which begin with the letter K are used to stop or kill that same service.

That leaves us with just the numbers. These are used to define the order in which the scripts are run. When the files are listed, they automatically appear in numerical order. In this way, the system can ensure the scripts are run in the correct order. For example, you do not want to start the Apache Web server before you start the network. Therefore, the linked used to start the network is S05network whereas the link used to start Apache is S20apache as S05 comes before S20 no matter what comes afterwards.

Note also, the same applies when the system shuts down. K20apache is used to shut down the Apache server and K40network is used to shut down network. As in the first case, the network is not shutdown until after Apache has.

It is interesting to note that this system could work even if the name of the link consisted of just S or K and the appropriate number. That is, it would still work if the link told us nothing of the service being started. There is actually more to it than making things simpler for us non-computers. Having the names at the end allows the system to avoid unnecessary the unnecessary stopping and starting of the various services. When a lower level is entered, only those of services are started which were not started in previous run level. When leaving a run level, the only services that are stopped are those that are not started in the new level.

Let's look at an example. In the directory /etc/init.d/rc3.d (for run level 3), there are links used to both start and stop the network. However, this means the network will always be re-started when moving from run level 1 to run level 3. This also means the network will always be stopped when moving from run level 3 to run level 1. On the other hand, both links exist in rc.2 (for run level 2). Therefore, when leaving either run level 2 or 3 and moving to the other, the network is not stopped as there is a start link for it in the new run level. When entering the new run level, the network is not started, as there was already a start link for the previous level. However, in moving from a run level when network is running (e.g. 2,3 or 5) to run level 1, the network is stopped because there is no link to start the network in run level 1.

We're not done yet.

Since the links to both start and stop a service can be to the exact same file, the script needs some way of knowing whether it should start or stop the service. This is done by passing an argument to the script: start to start the service and stop to stop the service (simple, huh?). Inside each script, this argument is read (typically $1) and different activities are performed based on what the argument was.

Note that for many scripts, you can pass other arguments than just start and stop. For example, one common argument is restart. As its name implies, this is used to stop then start the service again, in other words, restart a running service. Many will also accept the argument status, which is used to deliver status information about that service.

# Default runlevel.

id:3:initdefault:

# System initialization.
si::sysinit:/etc/rc.d/rc.modules default
bw::bootwait:/etc/rc.d/rc.boot

# What to do in single-user mode.

~1:S:wait:/etc/rc.d/rc 1
~~:S:wait:/sbin/sulogin

l0:0:wait:/etc/rc.d/rc 0
l1:1:wait:/etc/rc.d/rc 1
l2:2:wait:/etc/rc.d/rc 2
l3:3:wait:/etc/rc.d/rc 3
l4:4:wait:/etc/rc.d/rc 4
l5:5:wait:/etc/rc.d/rc 5
l6:6:wait:/etc/rc.d/rc 6

Figure - Excerpts from the /etc/inittab file

On some systems, the scripts we have talked about so far are not the only scripts which are started when the system boots. Remember that init reads the inittab to find out what to do, so there are any number of things that "could" be started through the inittab, as compared to the rc-scripts. Even so, for people who are used to other versions of UNIX, the inittab looks pretty barren.

One type of script that is often run from the inittab deals with system initialization. For example, the boot script, which is found directly in /sbin/init.d. The entry in the inittab might look like this:

si:I:bootwait:/sbin/init.d/boot

The run level this script runs in is "I", which is not a traditional run level, but used by some distributions (i.e. SuSe) to indicate system initialization. However, because the action is bootwait, the run-level field is ignored. Bootwait means that this entry will be processed while the system boots, and init will wait until the command or script has completed.

In this case, the script is /sbin/init.d/boot, which performs basic system initialization such as starting the bdflush daemon (which writes dirty buffers to the disk), checking the filesystems (with fsck), mounting filesystems, starting the kernel module daemon (kerneld), and many other things. Other versions (as well as other UNIX dialects) may have several different entries in the inittab that combine to do the same work as the /sbin/init.d/boot script under SuSe Linux.

The counterpart to the /sbin/init.d/boot script /sbin/init.d/halt. These are the procedures that are carried out when the system is brought down. In general, these are the reverse of the procedures in the boot scripts, such as stopping kerneld and unmounting filesystems.

SuSe also uses the system configuration file /etc/rc.config. This file contains a large number of variables that are used to configure the various services. Reading this file and setting the variables is one of the first things done by the script /sbin/init.d/rc. The counterpart to this file on Caldera is /etc/syconfig/daemons. Instead of a single configuration file, you will find separate files for a number of different daemons.

Creating your own init scripts

Sometimes the scripts your particular distribution provides are not sufficient and you need to add your own. On a number of systems where I have needed to add my own system services, I have needed to create my own init scripts. The method that works on any system is to simply follow the conventions used by your distribution.

SuSe has realized the need for creating your own init scripts, so has provided a template for you. This is the file /sbin/init.d/skeleton and as its name implies, is a "skeleton" init script. In its default state, this is a completely runnable init script. At the same time it is completely useless as there is no daemon behind it. Instead, you simply uncomment the lines you need, change the name of the daemon or service and you are ready to run.


Thursday, February 03, 2005

Everything you wante to know about ISO images in Linux

Tuesday, February 01, 2005

How it all began

So I thought to myself, if Douglas Adams could write a whole book based mostly on a couch getting stuck in a stairway then why can't I start a blog about things that I learn while windsurfing, playing the piano, building a jungle gym, making a movie, baking a cheescake or sweating away trying to get all my favourite music and video editing programs working on Ubuntu Linux.