Using a Manifest to Recreate your Linux System Selectively

Last week, I had finally had enough of not being able to hibernate my computer.  There was enough “chaff’ and weird things happening.

I did realize that I could create a list of everything I had, and then get Linux to import that list and reinstall all my programs.

That would be my Manifest.

I did it knowing that I could be reintroducing the problem that I created with the old system.

I was right.  So I did it over, selectively.

And it worked.  Hibernate and video crashes were problems, and after 17 consecutive hibernate cycles over two days of active use, I’d say I am done.

This was a whole lot simpler.  You see, this scary Manifest thing is nothing more than a text file that is generated within “Synaptic” that contains all the markings of the programs that I installed over the 7 years that I had that Linux install.

I went through that file and deleted everything that I did not expressly know what that particular program was, or anything I knew I did not want.

Easy except the file was in chronological order or … well, lets just pretend it was and leave it at that.  Basically it can be sorted in alphabetical program order simply.

One line in Terminal, just like everything in Linux, would solve it.

Assuming the Manifest is called /home/bill/Desktop/Manifest.txt

In Terminal, issue this command string on one line:

cat /home/bill/Desktop/Manifest.txt | sort > /home/bill/Desktop/SortedManifest.txt

Now you’re in alpha order, and it makes it easier.

I did delete anything that started “lib” as well as KDE, gnome, and mate since I strongly prefer XFCE to all of those.  My choice, no big deal

I simply edited the file in Mousepad, and deleted all things I did not want.

If you want the long form description of all of this, Last Week’s Post is at this link.  However the short form is here:

1) on original install create a Manifest within Synaptic Package Manager.

a) open synaptic

b) Select File, Save Markings As

c) navigate to the place you want to store this file, and give it a name.

d) Tick the box “Save full state, not only changes”

e) Click Save.

2) Verify that your manifest is on removeable media.

3)  Remove any unwanted programs from the Manifest

4) save your important files from the operating system on removable media
/etc/samba/smb.conf,

/etc/apt/sources.list,

the Manifest file

5) Install a fresh copy of your Debian Based operating system on the destination computer.

Debian, *Ubuntu, Linux Mint, whatever…

6) Get the destination computer “up to date” and stable.

7) compare and manually update your /etc/apt/sources.list file from the original computer

copy the installed version to a save file

I copied my own from the original computer in its place and updated

then you will need to update the PGP keys for one or more added such as http://www.deb-multimedia.com

8) install the manifest by

a) open synaptic

b) Select File, Read Markings

c) find and open the manifest.txt file

d) click open

e) verify needed markings have been imported into Synaptic, and click Apply.

f) there will be additional libraries incorporated into your install list due to any new dependencies.

8) you’re done.  Verify everything is OK.  Live with it for a while.

You will want to add in programs like libdvdcss to allow DVDs to play, Samba to share files, but these things will need to be done individually.

9) File Sharing.  I used the Debian Wiki entry at https://wiki.debian.org/SambaServerSimple

a) apt install samba samba-client

b) edit /etc/samba/smb.conf  – or put the one in from the old computer assuming you had it working.

c) add your samba users:  smbpasswd -a USERNAME

replace USERNAME with the correct name, and it will ask you for the password

d) restart Samba:

    # /etc/init.d/samba restart
    or, if you are using systemd
    # /usr/sbin/service smbd restart

Cloning a Hard Drive With Linux

Yeah well calling it Linux means I most likely lost 97% of the market.


Windows people don’t realize that there is a painless way to get their windows computer to do some of this stuff – a Live Linux Distribution like Ubuntu.  If you get a live disc working, you can copy this shell into it, then follow the instructions.  It should work.

Mac people may even be able to run this natively.

Maybe.  Depends if PV is Mac Friendly, if not, convert the PV line to a copy of your choice.


A Live Linux can be “burned” to a USB stick or to a DVD and your computer can be booted from that.


And now you know!


But none the less…

What this is basically is my own shell.  I use this to completely back up my computer.  All the drive specifications are found and known, and do not change.

I run fdisk -l as root and use the information in there to edit the shell script to change things as needed.

This assumes that you know what your drive devices are, are willing to edit a shell script to make your own changes as is, then have an external USB hard drive slightly larger than your boot device.  My boot device is /dev/sda and most likely yours is as well.

This assumes that you have a second drive sitting in your chip reader.  If not, you can comment out the line that copies it to the hard drive.

This assumes that you have room enough to do everything.

I am doing this on Debian Linux, however the commands here are so very generic that you should be able to run this on most “full” distributions of Linux.  Debian, Ubuntu, Linux Mint, Centos, Fedora and the like come to mind.

Standard Internet Warranty – I make no warranties and it is at your own risk.  If you lose data, it is on you.  I take zero responsibilities for any miscoding or changing or whether a magic dragon comes out of the skies and takes you onward to valhalla.  Really.  None at all.

I will say that I ran this exact shell this morning and it worked for me.  You WILL have to change the file specifications to fit.   

Finally:

  • My boot drive is a 240gb SSD with about 120gb free.
  • My chip has about 12 gb worth of data on it.
  • Debian thinks that the chip is called “128GB” and it typically comes up in the file manager (thunar) on /media/bill/128 GB/

Prerequisites:

Installed versions of

How it runs:

  • This must be run as Root in Terminal.
  • This will pause after each step with an OK message in the Dialog box.
  • For me, the entire shell runs in about 2 hours on my i7 laptop with a USB 2.0 external hard drive.

First the shell in its entirety through to the end comment:

#! /bin/bash


#backup.sh from http://www.ramblingmoose.com

dialog –no-lines –title ‘Run This As Root’ –msgbox ‘This shell will backup SDA to SDB\nYou must click OK after each step so watch this.\nYour Disaster Recovery will thank you!’ 10 70

dialog –no-lines –sleep 3 –title “update your sources” –prgbox “apt-get -y update” 10 70
dialog –no-lines –sleep 3 –title “update your software” –prgbox “apt-get -y upgrade” 10 70
dialog –no-lines –sleep 3 –title “update your distribution” –prgbox “apt-get -y dist-upgrade” 10 70

arg1=”‘/media/bill/128 GB'”

dialog –title “copying the chip to the drive” –prgbox “cp -avr $arg1 /home/bill/128GB” 10 70

(pv -n -i 2 /dev/sda > /dev/sdb) 2>&1 | dialog –title “Backup SDA to SDB” –gauge ‘Progress…’ 7 70

dialog –title ‘Message’ –msgbox ‘Cloning is done, click ok to clean up and end’ 5 70

dialog –no-lines –sleep 3 –title “Removing the copy of the chip” –prgbox “rm -r /home/bill/128GB” 10 70 

dialog –no-lines –sleep 3 –title “Synchronize your drives” –prgbox “sync” 10 70
#end backup.sh

To actually use that mess…

  • Copy the entire text and paste it into your favorite text editor.
  • Save the file with a “.sh” extension somewhere you will be able to get to it – in your path.
  • Change the mode to executable – chmod 0770 backup.sh
  • Change the owner to root.  You never want to use this as a regular user – chown root backup.sh
  • Change the group to root.  chgrp root backup.sh
  • Run the shell as root: sudo ./backup.sh

Now, each line in excruciating detail!

—- Run the programs using bash interpreter

#! /bin/bash

—- I’m signing my work here

#backup.sh from http://www.ramblingmoose.com

—- This puts up a message box

dialog –no-lines –title ‘Run This As Root’ –msgbox ‘This shell will backup SDA to SDB\nYou must click OK after each step so watch this.\nYour Disaster Recovery will thank you!’ 10 70

—- The next three steps gets your distribution to date.  Don’t want this, comment it out

dialog –no-lines –sleep 3 –title “update your sources” –prgbox “apt-get -y update” 10 70
dialog –no-lines –sleep 3 –title “update your software” –prgbox “apt-get -y upgrade” 10 70
dialog –no-lines –sleep 3 –title “update your distribution” –prgbox “apt-get -y dist-upgrade” 10 70

—- Store the directory that Linux mounts the chip to in “arg1”  If no chip to backup you can comment this.

arg1=”‘/media/bill/128 GB'”

—- Wrap the actual work of copying the chip out to a dialog box.  The flags “-avr” say copy the whole drive in $arg1 recursively to the destination.  If no chip to copy, comment this line.

dialog –title “copying the chip to the drive” –prgbox “cp -avr $arg1 /home/bill/128GB” 10 70

—- This line does the real work.  Now that you copied your chip out to the hard drive, clone the actual hard drive.  The flags on pv tell it to report to stdout the percentage of work done so that dialog can show a pretty gauge.  Ahh, so pretty!

(pv -n -i 2 /dev/sda > /dev/sdb) 2>&1 | dialog –title “Backup SDA to SDB” –gauge ‘Progress…’ 7 70

—- Copy is done, it is time to clean up message

dialog –title ‘Message’ –msgbox ‘Cloning is done, click ok to clean up and end’ 5 70

—- remove the data that you copied from the chip from the hard drive to be neat. if no chip, comment this out.

dialog –no-lines –sleep 3 –title “Removing the copy of the chip” –prgbox “rm -r /home/bill/128GB” 10 70 

—- Your work is done, make sure you flush your cache by doing a “sync”.

dialog –no-lines –sleep 3 –title “Synchronize your drives” –prgbox “sync” 10 70  

#end backup.sh