Using a Manifest to Recreate your Linux System Selectively

Last week, I had finally had enough of not being able to hibernate my computer.  There was enough “chaff’ and weird things happening.

I did realize that I could create a list of everything I had, and then get Linux to import that list and reinstall all my programs.

That would be my Manifest.

I did it knowing that I could be reintroducing the problem that I created with the old system.

I was right.  So I did it over, selectively.

And it worked.  Hibernate and video crashes were problems, and after 17 consecutive hibernate cycles over two days of active use, I’d say I am done.

This was a whole lot simpler.  You see, this scary Manifest thing is nothing more than a text file that is generated within “Synaptic” that contains all the markings of the programs that I installed over the 7 years that I had that Linux install.

I went through that file and deleted everything that I did not expressly know what that particular program was, or anything I knew I did not want.

Easy except the file was in chronological order or … well, lets just pretend it was and leave it at that.  Basically it can be sorted in alphabetical program order simply.

One line in Terminal, just like everything in Linux, would solve it.

Assuming the Manifest is called /home/bill/Desktop/Manifest.txt

In Terminal, issue this command string on one line:

cat /home/bill/Desktop/Manifest.txt | sort > /home/bill/Desktop/SortedManifest.txt

Now you’re in alpha order, and it makes it easier.

I did delete anything that started “lib” as well as KDE, gnome, and mate since I strongly prefer XFCE to all of those.  My choice, no big deal

I simply edited the file in Mousepad, and deleted all things I did not want.

If you want the long form description of all of this, Last Week’s Post is at this link.  However the short form is here:

1) on original install create a Manifest within Synaptic Package Manager.

a) open synaptic

b) Select File, Save Markings As

c) navigate to the place you want to store this file, and give it a name.

d) Tick the box “Save full state, not only changes”

e) Click Save.

2) Verify that your manifest is on removeable media.

3)  Remove any unwanted programs from the Manifest

4) save your important files from the operating system on removable media
/etc/samba/smb.conf,

/etc/apt/sources.list,

the Manifest file

5) Install a fresh copy of your Debian Based operating system on the destination computer.

Debian, *Ubuntu, Linux Mint, whatever…

6) Get the destination computer “up to date” and stable.

7) compare and manually update your /etc/apt/sources.list file from the original computer

copy the installed version to a save file

I copied my own from the original computer in its place and updated

then you will need to update the PGP keys for one or more added such as http://www.deb-multimedia.com

8) install the manifest by

a) open synaptic

b) Select File, Read Markings

c) find and open the manifest.txt file

d) click open

e) verify needed markings have been imported into Synaptic, and click Apply.

f) there will be additional libraries incorporated into your install list due to any new dependencies.

8) you’re done.  Verify everything is OK.  Live with it for a while.

You will want to add in programs like libdvdcss to allow DVDs to play, Samba to share files, but these things will need to be done individually.

9) File Sharing.  I used the Debian Wiki entry at https://wiki.debian.org/SambaServerSimple

a) apt install samba samba-client

b) edit /etc/samba/smb.conf  – or put the one in from the old computer assuming you had it working.

c) add your samba users:  smbpasswd -a USERNAME

replace USERNAME with the correct name, and it will ask you for the password

d) restart Samba:

    # /etc/init.d/samba restart
    or, if you are using systemd
    # /usr/sbin/service smbd restart

Cloning a Hard Drive With Linux

Yeah well calling it Linux means I most likely lost 97% of the market.


Windows people don’t realize that there is a painless way to get their windows computer to do some of this stuff – a Live Linux Distribution like Ubuntu.  If you get a live disc working, you can copy this shell into it, then follow the instructions.  It should work.

Mac people may even be able to run this natively.

Maybe.  Depends if PV is Mac Friendly, if not, convert the PV line to a copy of your choice.


A Live Linux can be “burned” to a USB stick or to a DVD and your computer can be booted from that.


And now you know!


But none the less…

What this is basically is my own shell.  I use this to completely back up my computer.  All the drive specifications are found and known, and do not change.

I run fdisk -l as root and use the information in there to edit the shell script to change things as needed.

This assumes that you know what your drive devices are, are willing to edit a shell script to make your own changes as is, then have an external USB hard drive slightly larger than your boot device.  My boot device is /dev/sda and most likely yours is as well.

This assumes that you have a second drive sitting in your chip reader.  If not, you can comment out the line that copies it to the hard drive.

This assumes that you have room enough to do everything.

I am doing this on Debian Linux, however the commands here are so very generic that you should be able to run this on most “full” distributions of Linux.  Debian, Ubuntu, Linux Mint, Centos, Fedora and the like come to mind.

Standard Internet Warranty – I make no warranties and it is at your own risk.  If you lose data, it is on you.  I take zero responsibilities for any miscoding or changing or whether a magic dragon comes out of the skies and takes you onward to valhalla.  Really.  None at all.

I will say that I ran this exact shell this morning and it worked for me.  You WILL have to change the file specifications to fit.   

Finally:

  • My boot drive is a 240gb SSD with about 120gb free.
  • My chip has about 12 gb worth of data on it.
  • Debian thinks that the chip is called “128GB” and it typically comes up in the file manager (thunar) on /media/bill/128 GB/

Prerequisites:

Installed versions of

How it runs:

  • This must be run as Root in Terminal.
  • This will pause after each step with an OK message in the Dialog box.
  • For me, the entire shell runs in about 2 hours on my i7 laptop with a USB 2.0 external hard drive.

First the shell in its entirety through to the end comment:

#! /bin/bash


#backup.sh from http://www.ramblingmoose.com

dialog –no-lines –title ‘Run This As Root’ –msgbox ‘This shell will backup SDA to SDB\nYou must click OK after each step so watch this.\nYour Disaster Recovery will thank you!’ 10 70

dialog –no-lines –sleep 3 –title “update your sources” –prgbox “apt-get -y update” 10 70
dialog –no-lines –sleep 3 –title “update your software” –prgbox “apt-get -y upgrade” 10 70
dialog –no-lines –sleep 3 –title “update your distribution” –prgbox “apt-get -y dist-upgrade” 10 70

arg1=”‘/media/bill/128 GB'”

dialog –title “copying the chip to the drive” –prgbox “cp -avr $arg1 /home/bill/128GB” 10 70

(pv -n -i 2 /dev/sda > /dev/sdb) 2>&1 | dialog –title “Backup SDA to SDB” –gauge ‘Progress…’ 7 70

dialog –title ‘Message’ –msgbox ‘Cloning is done, click ok to clean up and end’ 5 70

dialog –no-lines –sleep 3 –title “Removing the copy of the chip” –prgbox “rm -r /home/bill/128GB” 10 70 

dialog –no-lines –sleep 3 –title “Synchronize your drives” –prgbox “sync” 10 70
#end backup.sh

To actually use that mess…

  • Copy the entire text and paste it into your favorite text editor.
  • Save the file with a “.sh” extension somewhere you will be able to get to it – in your path.
  • Change the mode to executable – chmod 0770 backup.sh
  • Change the owner to root.  You never want to use this as a regular user – chown root backup.sh
  • Change the group to root.  chgrp root backup.sh
  • Run the shell as root: sudo ./backup.sh

Now, each line in excruciating detail!

—- Run the programs using bash interpreter

#! /bin/bash

—- I’m signing my work here

#backup.sh from http://www.ramblingmoose.com

—- This puts up a message box

dialog –no-lines –title ‘Run This As Root’ –msgbox ‘This shell will backup SDA to SDB\nYou must click OK after each step so watch this.\nYour Disaster Recovery will thank you!’ 10 70

—- The next three steps gets your distribution to date.  Don’t want this, comment it out

dialog –no-lines –sleep 3 –title “update your sources” –prgbox “apt-get -y update” 10 70
dialog –no-lines –sleep 3 –title “update your software” –prgbox “apt-get -y upgrade” 10 70
dialog –no-lines –sleep 3 –title “update your distribution” –prgbox “apt-get -y dist-upgrade” 10 70

—- Store the directory that Linux mounts the chip to in “arg1”  If no chip to backup you can comment this.

arg1=”‘/media/bill/128 GB'”

—- Wrap the actual work of copying the chip out to a dialog box.  The flags “-avr” say copy the whole drive in $arg1 recursively to the destination.  If no chip to copy, comment this line.

dialog –title “copying the chip to the drive” –prgbox “cp -avr $arg1 /home/bill/128GB” 10 70

—- This line does the real work.  Now that you copied your chip out to the hard drive, clone the actual hard drive.  The flags on pv tell it to report to stdout the percentage of work done so that dialog can show a pretty gauge.  Ahh, so pretty!

(pv -n -i 2 /dev/sda > /dev/sdb) 2>&1 | dialog –title “Backup SDA to SDB” –gauge ‘Progress…’ 7 70

—- Copy is done, it is time to clean up message

dialog –title ‘Message’ –msgbox ‘Cloning is done, click ok to clean up and end’ 5 70

—- remove the data that you copied from the chip from the hard drive to be neat. if no chip, comment this out.

dialog –no-lines –sleep 3 –title “Removing the copy of the chip” –prgbox “rm -r /home/bill/128GB” 10 70 

—- Your work is done, make sure you flush your cache by doing a “sync”.

dialog –no-lines –sleep 3 –title “Synchronize your drives” –prgbox “sync” 10 70  

#end backup.sh

The Netbook Server – How to Actually Share Part Of The Hard Drive

First, you installed Linux to a RaspberryPi or a Netbook, or whatever you had on hand.

Second, you made it so you could look into that machine from anywhere on your network.

If all you wanted was a taste of how to run Linux and have fun with all those free goodies there, you could have stopped.  Now I’m going to show you how to take a part of the hard drive (a folder) and share it out to the network.

Why?

So you can copy your pictures/recipes/important crap somewhere else.

So you can back up your computer across the network.

So you can brag to the co-workers that you have a proper Linux Home Server and sound like you know what you’re doing.

Well the deal is that it took me a half hour to do this last night.  I was distracted by what was on the TV so it would have taken less time.

I did this on a RaspberryPi first.

 

Since my instructions were written there I then repeated the steps on my Netbook running Debian, so the instructions work.  It also works on anything derived from Debian Linux, so that if you have found this article using Linux Mint, Ubuntu, or any of the other derivatives from the Debian Family, you SHOULD be able to get this working with very little fuss.

If you are familiar with Linux and the way things work, you’re used to finding instructions that promise to do something, get totally frustrated that the instructions are geeked out, and then realize that while it’s working you don’t actually understand WHY things are done this way.

I’m going to attempt to do it differently.  This way when I have to look at it later, I can look at my own B.S. here and say “Oh yeah, I remember this”.

The information you need:

1) Your sign on name – this will be written assuming you are “bill”.  Just change that to your own name from when you created the machine.

2) Your “root” and regular user (bill) Passwords.  

3) The name you gave the computer when you installed Linux.  It could be pi or rudolph or any other name you came up with.  

I will make assumptions and try to explain it all away.  Don’t worry, I followed these same steps last night and the server now “serves” files out to the network.  As long as your network has a firewall, your stuff is safe.

Get the machine “up to date”:

  1. Start your Terminal from the start menu.
  2. su and hit enter – Get “root” by giving it the root password.
  3. apt-get update – pull down all the headers of new stuff since the last time you got on the machine
  4. apt-get upgrade  – actually get all the upgraded software

Answer yes or Y to the prompt asking if you really want to update things, go make yourself something from the kitchen and come back in a bit.  It may take time.  There are always updates.  But if you never make changes to your “Repositories” on Debian or Raspbian, you are safe and free from any nasty viruses.

 

 

Get the Server Software Installed:

You’ll be shocked how little has to be done here.   The server software is called SAMBA.  You know, like the great music from Brazil?   The current name has been made less fun – CIFS.  People tend to say it as “Siffs”.

Geeks.

One line gets the software.  In terminal from the last part, as root enter the next line:

apt-get install samba samba-common-bin

Configure the Server Software:

You have to roll up your sleeves here.  You are actually going to change a text file, but I’m going to give you the information.  Remember – I am entering it as “bill”.  If you are on RaspberryPi, your regular user will most likely be “pi”.    Change “bill” to what you need it to be.

 

Second, a comment starts with a hash tag.  #

 

1) In terminal where you are signed on as “root”, enter the following line to get into the “Nano” editor:

 

nano /etc/samba/smb.conf  

2) Find the line with “wins support” and change the line to read:

wins support = yes

3) Find “Share Definitions”.  You are going to enter in a block of text.  Remember to change the path from “bill” to match your login name.

 

[Downloads]
comment = Downloads Directory
path = /home/bill/Downloads
browseable = no

    writeable = yes
only guest = no
create mask = 0770
directory mask = 0770
public = no

 

#browseable limits logins to only see this directory and what is created there.  “yes” shares everything.

 

4) ctrl+x to exit, type y to save the file, then enter to get yourself back out to the root terminal prompt.

5) add a Samba user to be able to share that directory.  In terminal enter the following:

smbpasswd -a bill

Enter in a password, then enter it in again.  This is the password you will need to have to be able to get at the files from out on the network on another machine.  You will log in as (bill) and (password) from that other machine when you try to get there using File Manager.  Same thing with Mac or Linux.  They all need that password.

Write down your password.  I recommend using the same as your regular user password.  If you made them all the same as the Root password, well that may be easier.  You can also leave it blank, but I do not recommend that.  In fact, forget I mentioned it (or not…).

6) restart the computer


At this point, the netbook server is now visible on the network.  It is sharing the default login’s “Downloads” directory.

If you go into File Manager in Windows, you will be able to get to the files that are stored in the netbook’s /home/bill/Downloads directory from any other machine on the network as long as you know the default user’s login.  It will ask you for user and password.

If you have followed this, you can use the computer’s name from when you created it.  I now have two servers “rudolph” for the netbook and the raspberryPi.  If computer names aren’t your thing, you can also find them via IP addresses.

But at this point you have a functioning File Server.

You’re done.  Next time it’s getting this machine to serve out files from an external drive that you plugged in.