How do you backup your own stage4 environment?

[asong26]

I've posted this in the Gentoo forums, but did not get a response yet. Perhaps, I'll get a faster response here from Cyanide??

Here's what I've posted earlier....

How do you create your own stage 4 tar ball?

I found this file here: ftp://blinkeye.ch/gentoo/mkstage4.sh

The problem is, the script parameters on many points are not compatible with the official stage 4 releases (files and directory structures...)

Here's what I would like to do:

I would like to have a tar ball available on my USB drive with all of the modifications that are already made to files such as make.conf, fstab, including the samba installation as default. In case I have re-setup linux, it would be so much to have a restore point without having to go through all of the config file mods that I usually end up having to do... Any suggestions?

As I am at a learning stage of Gentoo (even linux in general), I am constantly tinkering with the kernel, hardware settings, compiler settings, etc... as 'root' to make it easier. Of course, being a root allows you to do anything, but you can very easily break your system this way. But I believe this is a big part of learning process for me. I've re-installed Gentoo 3 times now. And I am learning more and more each time I break something

Since my experiments require accessbility to apps such as firefox, vlc, and samba, It's a painfully long process to bring the system back to the test stage again and and again. Not to mention all of the config file mods.

Can you help? I would like to do a clean install with the mods that I need, and back that up, so that I can break the system over and over again until I know how I want my system to be....

[ikarius]

Originally Posted by asong26

I've posted this in the Gentoo forums, but did not get a response yet. Perhaps, I'll get a faster response here from Cyanide??

Here's what I've posted earlier....



As I am at a learning stage of Gentoo (even linux in general), I am constantly tinkering with the kernel, hardware settings, compiler settings, etc... as 'root' to make it easier. Of course, being a root allows you to do anything, but you can very easily break your system this way. But I believe this is a big part of learning process for me. I've re-installed Gentoo 3 times now. And I am learning more and more each time I break something

Since my experiments require accessbility to apps such as firefox, vlc, and samba, It's a painfully long process to bring the system back to the test stage again and and again. Not to mention all of the config file mods.

Can you help? I would like to do a clean install with the mods that I need, and back that up, so that I can break the system over and over again until I know how I want my system to be....

I don't know what you are trying to do exactly, so, maybe if you could answer this question, I might be able to help you more. What are you trying to backup ? The whole stage4 (which means the whole directory structure of your root) or only the configuration files located in /etc/ ? The way you were saying it in your previous post, it seems that it was only the configuration files, so, this is why I would like you to clear this out for me as I could help you.

[asong26]

Sorry for the confusion.

To simply, I am trying to create a tarball that can be re-installed later on.

What I want in the tar ball is the official stage4 64bit tarball, and all of the modifications that I made w/ all of the apps that I've installed and uninstalled.

For an example. I want a tar ball with with KDE, no fluxbox, and all of the hardware settings changes that I've made in fstab, make.conf, /etc/portage/package.use, /etc/portage/package.keywords, and not to mention my user settigs, home directory. Just about everything that is needed to restore my environment.

I guess in Windows terms, I am trying to create a ghost image....
-= Double Post =-

Originally Posted by asong26

Sorry for the confusion.

To simply, I am trying to create a tarball that can be re-installed later on.

What I want in the tar ball is the official stage4 64bit tarball, and all of the modifications that I made w/ all of the apps that I've installed and uninstalled.

For an example. I want a tar ball with with KDE, no fluxbox, and all of the hardware settings changes that I've made in fstab, make.conf, /etc/portage/package.use, /etc/portage/package.keywords, and not to mention my user settigs, home directory. Just about everything that is needed to restore my environment.

I guess in Windows terms, I am trying to create a ghost image....

On second thought, if there is a way to create a ghost image, so that I can skip the entire partitioning / formatting process, it would be great!

[cyanide]

Have a look here:
http://gentoo-wiki.com/HOWTO_Custom_Stage4

Read every step and tell me if you find anything which you can't understand

[ikarius]

Originally Posted by asong26

Sorry for the confusion.

To simply, I am trying to create a tarball that can be re-installed later on.

What I want in the tar ball is the official stage4 64bit tarball, and all of the modifications that I made w/ all of the apps that I've installed and uninstalled.

For an example. I want a tar ball with with KDE, no fluxbox, and all of the hardware settings changes that I've made in fstab, make.conf, /etc/portage/package.use, /etc/portage/package.keywords, and not to mention my user settigs, home directory. Just about everything that is needed to restore my environment.

I guess in Windows terms, I am trying to create a ghost image....

On second thought, if there is a way to create a ghost image, so that I can skip the entire partitioning / formatting process, it would be great!

Alright.

First, if you wish to create a tarball of your whole hard drive you can use this command:
tar -cjf stage4.tar.bz2 /

This will put in stage4.tar.bz2 the content of the whole hard drive. This is the easiest and fastest solution. Then, all you need to do, unfortunatly since you wanna avoid it, is to redo the installation using this tarball.

For the "ghost image" probably someone can help you with this, as I never performed this in linux. I used to do a tarball of the whole hard drive to preserve any modifications I did.

Of course, to create the tarball you will need to be logged as root.

Hope I could help you more about the ghost image, but unfortunatly, like I said, I never performed this.

-= EDIT =-

Thanks Cyanide, hehe, this will probably help him more than I can.

[asong26]

Originally Posted by cyanide

Have a look here:
http://gentoo-wiki.com/HOWTO_Custom_Stage4

Read every step and tell me if you find anything which you can't understand

cyanide: That is where I looked first. My concern was that the file structure that this guide refers to doesn't exactly match that of what we have installed on PS3. For instance, the above ftp link that Iprovided is based on the wiki you are referring to. There are many directories that these shell scripts have in them such do not exist on PS3. Therefore, when I attempt to execute, I get an error such as "/usr/srv/xxx doesn't exist. Maybe the PS3 install tar image is a bit different than what's on the wiki. Read my first post, and look at the file that I am pointing to. You'll see that the directory strucutres that the script is copying does not match with the directory structure created via official stage 4 tarball. Can you help me to understand what part of that script I need to modify to make it work on my environment? I can figure out most of the stuff, but only thing I don't know is stuff like where's my /usr/src directory, etc..., I don't have that particular direcotry on my PS3. There must be an equivalent directory, but where?

Thanks.
-= Double Post =-

First, if you wish to create a tarball of your whole hard drive you can use this command:
tar -cjf stage4.tar.bz2 /

Thanks, I'll try that!
-= Double Post =-
One more question regarding the tar. How do you access the device where the tar will be sitting on. In this case, my external USB which in my /etc/fstab shows as dmesg shows it as /dev/sdf1.

If I want to do a clean install using the liveCD to create a new partiton and format, I would assume that I'd have to mount the usb drive from the kboot prompt correct?

If so, will this command work from the kboot?

mount /dev/sdf1 /mnt/gentoo

And lastly, after I un-tar the tarball using tar -xjpf mybackup.ttar.bz2, I can just reboot and everything should be restored right?
-= Double Post =-
Trying now.....

Should I be getting any errors? I am getting bunch of Cannot open: permission denied errors. I am logged in a root.

Namely: /sys/devices/Platform/ps3fb.0
/sys/devices/Platform/ps3pf_sector

Is it ok to ignore this?
-= Double Post =-
Nevermind.... It crapped out.....

[dwmyers]

mount won't work from kboot. mount is a Linux command, not a kboot command.

If you have ssh working, you can pipe your archive to a remote server.

Examples using cpio (yet another archiving utility):

started on the ps3:

cd / ; find . -depth -print | cpio -ocB | ssh my-account@remote-machine.com "dd ibs=5120 of=ps3-archive-file.cpio"

There are similar tricks using tar as well. See for example:

http://www.bigbold.com/snippets/posts/show/1337

Now, if you have an image of your machine saved, you rebuild it as follows:

1. Boot from a live cd.
2. mount your PS3 file system in the live CD environment.
3. format as needed.
4. use the tarball or cpio archive to recover your old stuff.

The live CD does not have to be the same distro as the archive you recover.

David.

[asong26]

Ok,

Now we're getting somewhere. I was able to load up the livecd, and start the ssh session.

I was also able to create a new mount point ( /mnt/usb ) using the command:

mkdir /mnt/usb
mount /dev/sdf1 /mnt/usb

It worked. So, now I mount the /dev/sda1 to /mnt/gentoo, and my mount points are:

PS3 hard drive = /mnt/gentoo
USB Hard Drive = /mnt/usb

At this point I have both PS3 linux partition and my external usb drive both mounted. I tested both those directories, and confirmed that the mount wa successful.

At this point, how do I tar up the ps3 hard hard drive and put that tar ball on my usb? I want to make absolutely certain that the built tar ball will decompress right onto the /dev/sd1 as it should so that it'll boot.

Would it be: tar -cjf /mnt/usb/mybackup.tar.bz2 /dev/sda1 ?
Or

Is it tar /mnt/usb/mybackup.tar.bz2 /mnt/gentoo

??

I think we're almost there....

[dwmyers]

Of the two commands you presented, the second is more accurate. /dev/sda1 would present the bit image of the drive. tar is file oriented.

Correction -- /dev/sda would present the bit image of the drive. /dev/sda1 is simply a partition.

By the way, I took a peek at the Wikipedia (great Linux resource, btw; it's becoming a better set of man pages) and it noted that 'dd' is a ghosting utility for Linux/Unix. However, you'll get an exact image of your disk, and in all honesty, it's not as useful as a tarball or cpio utility.

Some useful Wikipedia links:

http://en.wikipedia.org/wiki/Disk_cloning
http://en.wikipedia.org/wiki/Cpio
http://en.wikipedia.org/wiki/Dd_%28Unix%29
http://en.wikipedia.org/wiki/Tar_%28file_format%29

I think the syntax you want is:

tar -cjf /mnt/usb/mytarfile.tar.bz2 /mnt/gentoo

By the way, if you're doing a hard drive to usb copy using a live disk, all local, you don't need ssh. ssh is only to copy the archive to another machine over the network.

Now, assuming you attached a usb drive that was larger than the PS3 drive, then you could "ghost" the drive with this:

dd if=/dev/sda of=/mnt/usb/disk.image

Understand that if you do this kind of thing, you've also copied the boot loader, the underlying superblocks and inodes, as well as the files you want. Tar archives can be installed on different sized disks, which may have totally different file systems. Tar archives are more versatile as an archive mechanism.

David.

[asong26]

Problem:

I did this: tar -cjf /mnt/usb/mytarfile.tar.bz2 /mnt/gentoo

Tar file got created OK. I reformatted the drive, created the partitions as it was before. I then mounted /dev/sda1 to /mnt/gentoo.

After that, I've copied over the created tar file to /mnt/gentoo/

I CD into /mnt/gentoo/ and applid this command:
tar -xjpf xxxxx.tar.bz2.

It untarred, but the problem the files are untarred at 2 extra directories below. My disk now looks like this:

-mnt
----gentoo
--------etc
--------usr
--------xxx
--------xxx

Basically, created 2 tar directory structure now contails /mnt/gentoo.

How do I fix this?

[dwmyers]

You clean up your disk and run the tar utility from the root directory

1. cd /mnt/gentoo
2. rm -rf *
or you can clean it up by reformatting the partition.
3. cd /
4. tar -xjf /mnt/usb/mytarfile

*before* doing this, confirm that the tar file is relative to root by doing..

tar -tjvf /mnt/usb/mytarfile

and look at the output. If you see lines like:

./mnt/gentoo/stuff

then you need to untar from the root directory

David.

[asong26]

You're the man! That worked, except for a minor detail peskiness...

After the files are all laid out, the kboot does not execute kboot.conf. I have to type in the boot parameters manually.

Even when I do, it stops in the middle saying that the root block device is not defined. I then type in /dev/sda1. It's the proceeds to boot. What could cause this?

The Kboot.conf file is clearly there....

[dwmyers]

It sounds as if the issue is in the boot block, or perhaps permissions on the kboot.conf file itself.

What do you see with 'ls -l /etc/kboot.conf' ?

If permissions were/are okay, I'd be tempted to reinstall the boot loader. It won't affect your other software.

One last thing.. if you can, post your /etc/kboot.conf. Maybe there is a problem with it as well.

David.

[asong26]

localhost / # ls -l /etc/kboot.conf
-rw-r--r-- 1 root root 1265 Jan 23 02:33 /etc/kboot.conf

If it's not the permissions, I have a hunch. If you look at the install script of the tar ball. there are these steps after the tarball has been untar-ed.
:

livecd ~ # mount -t proc none /mnt/gentoo/proc
livecd ~ # mount -o bind /dev /mnt/gentoo/dev
livecd ~ # cp /etc/resolv.conf /mnt/gentoo/etc

livecd gentoo # chroot /mnt/gentoo/ /bin/bash
livecd / # export PS1="(chroot) $PS1"
chroot) / # env-update && source /etc/profile
--- 'profiles/arch.list' is empty or not available. Empty portage tree?
>>> Regenerating /etc/ld.so.cache...

Does that make sense to you?

[dwmyers]

It looks as if you're physically moving your running environment to the HD, chrooting onto your mounted disk, and then the error message you get has to do with the package management tool of Gentoo.

I'm not running Gentoo and I can't help you much with portage. What I was trying to do is get you to the point you could reliably back up your disk.

David.

[asong26]

Originally Posted by dwmyers

It looks as if you're physically moving your running environment to the HD, chrooting onto your mounted disk, and then the error message you get has to do with the package management tool of Gentoo.

I'm not running Gentoo and I can't help you much with portage. What I was trying to do is get you to the point you could reliably back up your disk.

David.

David,

Kboot.conf now executes without a problem! In the Gentoo world, this is how the tarballs are installed and configured. I thought since I've done that durng the first install, that I don't have to do this anymore. I don't begin to understand the technicality of these steps, but it work beautifully.

Now I am going to perform a clean install, set everything up so that by default, I'll be able to connect to my windows pc drive, tidy up all of the config files, and then make a backup of this tar ball for future use.

This way, I can tinker all I want, and still have a quick recovery option that does not make me go through all of the config steps (painfully long time).

I appreciate the attention and help! I am for everyeon else who can appreciate a similar setup will value this thread, and your input!!