Is Anyone Writing Documentation Anymore?

So the last exercise I had done before Hurricane Irma hit was to take a computer and completely install a new operating system onto it.

I will be doing that again today, from my own instructions.

 

Sure, it’s a bit esoteric, some people are good at reading between the lines.

 

The problem with that is that you end up spinning your wheels and finding that something you assumed, you assumed wrong.

 

For the record, when I write documentation, I write it as I do it. That way I know it actually works.

 

It may have taken 26 steps once you had all the pieces, but if you had my hardware and the right software, you’d have a nice happy laptop running Debian. Thinkpad Laptop, X201 or fairly similar, although the version of Debian I used (Non-Free) was fairly liberal with getting what you need for many more laptops. Evil Wifi Drivers not withstanding.

 

The next step was to find documentation to install a web server.

 

The trick with installing complex software these days is that you basically have to find the right documentation. Or to be more precise, the correct documentation. Documentation that is complete and actually will work.

 

Oh and of course you personally have to read and understand what you are reading. No distractions allowed.

 

However, it is rare that you will find exactly the right documentation to do what you want. Often software is updated and that documentation you used two years ago to do that exact thing no longer works.

 

Highly common in the open source world, some very minor tweak will change where the files are and you are back online doing a search for what you were looking for.

 

In the consumer software world, you have a similar situation where the documentation was only partially updated since it was originally released. Think Windows XP vs Windows 8.1 vs Window 10. Things just moved around drastically within Windows itself let alone functionality.

 

I got “caught short” with trying to install a web server. Did it before. No problem. Since it is an open source project, you get what you pay for sometimes. Following the wrong guide I got the entire web server working. I just don’t have any passwords for anything.

 

As they say on a football field: Drop Back 5 and Kick.

 

That would be American Football. I never played Soccer, at least not for any length of time. All that running around annoyed me.

 

So at some point I’ll re-attempt that mess. FInd another tutorial that promises to install the LAMP stack and write down what I did.

Or I will find that one bug and fix it all. My choice, after all.

 

That is why I keep this blog. Many times I need to do something more than once. Create a Linux Web Server, save it off, then reproduce the results on a different computer a year later.

 

So when I post a long diatribe on how to do this and the other thing, I’m doing it for Future Me. So I don’t end up banging my head on a wall.

 

Like Today. I got it wrong. Happens. Time to start over.

 

For now, I’ll just go look for the football. Maybe the dog will chase it around the yard. Blow off some steam. Finish my Spanish for the day.

Try, Try again.

Advertisements

Debian Linux Stretch – Installing the Operating System

This process took me all of 20 minutes.  It will take you longer to read this blog article.

I have made this into a “recipe” format where you can follow things step by step since I wrote this as I was doing it.

It is wordy, but complete – and it worked for me, step by step.

I need a new server.   I had an old netbook that I used for a couple years to move files around.  It’s too slow for me now, I will eventually be updating this new install to include a file server and a web server.

The blog sits on Blogger and WordPress.  Two places.  I need to make the WordPress side look “better”.  I have a client that I had developed a website for and I am not happy with the way it looks.  So I want a playground to see how tough it will be to get it where I think it needs to be.

So why not build a server.   You can skip to the break if you want Just The Steps.

The background is that I use Debian Linux here for almost everything I do.  I have a Windows computer that I almost never use.  Microsoft has turned Windows into an unpleasant operating system where you are sending data back at every turn of what you are doing.  Who ever made the decision to grant themselves that should be fired, mocked, pilloried, set in the stocks, and …

Never mind, there’s Debian.  It’s stable, it is predictable, and because it isn’t spying on you, it runs faster and is much more secure.

Oh and it’s free.

And there is so much information about how to use it that it is insane!

And it’s secure.

And it lets me do everything I want.

And it has a long list of software that has everything I need.

And I can use it for web development, audio and video authoring, file servers, surfing, document processing, …

Get the picture?

Ok.  Surf https://www.debian.org and do a little reading.  Come on back when you’re done.  I’ll be here.

The philosophy is to give you (the user) what you need to get your things done, be stable, and stay out of the way.  It is a bit “spare” or “lightweight”.  The base operating system is familiar but a bit retro or stodgy looking.  It is also intensely configurable.

I mean INTENSELY.  You can change just about anything which is good because I am tailoring this for a moderate speed boost by using a display manager called XFCE.  My choice, you can choose anything else you want.  KDE, Gnome, XFCE, Mate, or Cinnamon.  Others are available and you can end up with something looking like a Mac, Windows 2000, or even just a blank command prompt for slower computers.

Some background info to be aware of:

Debian is a distribution of Linux that does its best to be as open and secure as possible.  The theory is that if you can modify the actual source code, it will be secure because thousands of eyes will be reviewing the program.  That also means that Debian does not include software that is not “open” and is Proprietary and “Non-Free”.  Wifi drivers are the worst offenders, and it is notorious that a Thinkpad will demand a Wifi driver.

The suggestion is that when you move to start the install process, you do so with the computer plugged in, and plugged into an ethernet connection so that it can find whatever drivers it needs.

If the install errors or “fails” it will put up a large notification saying it needs the driver.  That error message will tell you what driver you need, and you can do a search online to find it.  The drivers are always there, I have found, with mainstream hardware.  It means that I have to find the proper package and put it on a second memory stick or chip and let the install program find it.  Since all my computers are Thinkpad laptops, I have the drivers I need on an SDHC chip and an install just grabs it from there.


tl;dr – know your hardware and get the drivers before you start.  Look at Non-Free ISOs first.


Enough blather.  Just the steps.

1) Get the Live DVD Image.  It will allow you to burn the “ISO” to a DVD and boot from it directly or you can use a program to “burn” it to a USB stick and you can boot from that.

Direct Link: https://cdimage.debian.org/images/unofficial/non-free/images-including-firmware/9.1.0-live+nonfree/amd64/iso-hybrid/debian-live-9.1.0-amd64-xfce+nonfree.iso

Link to find “Non Free” versions with many more drivers installed:  https://cdimage.debian.org/images/unofficial/non-free/images-including-firmware/

Link For Other Versions:   https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/

Select the file that ends in “ISO” that you want.


2) Put the data onto your DVD or Memory Stick:

DVD – use your favorite burner software to write the DVD.

unetbootin – will write the ISO you just downloaded to a memory stick.  Make sure your stick is 2GB or larger.  Follow the specific instructions for your operating system for the program.

DD – for Linux and Mac, you can dd the ISO to the USB stick.  If that usb is on /dev/sdb and the ISO is renamed to debian.iso :

dd if=debian.iso of=/dev/sdb conv=noerror,sync

Once that is done, safely detatch your stick, or dismount your DVD and begin.

3) Boot from your media:

You will probably have to hit a key, typically F2 or F12 or Esc to get into the bios to tell the computer to boot from the external media.  I typically set my medias to prefer to boot from the stick or DVD drive so this happens automatically.  Since everybody has different computers, I’ll tell you to look for a way to set that up, a way to boot from media, or just say go for it.

4) Optional – Test your computer:

A Live DVD or USB Stick is a cool invention.  You basically are booting from the media and are able to run from it.  You have full control, if you know how to do anything in Linux, so you should be able to connect to the network, test your hardware, and even surf to a music site.   While that all sounds like it’s optional, it also makes sure that Linux understands your hardware.  That is not completely optional, but I’ll leave that to you.

5) Graphical Debian Installer:  

You tested the machine, or not and are committed to install.  Select this from the original Main Menu

6) Steps to Install: Steps A to Z will get you where you want to be!

Hit enter on the Graphical Debian Installer prompt.  It will present you with the following questions:

A) Select a Language – English is the default.  Change it to your preferred language

B) Select your Location – United States is the Default.  Change to your preferred area or “other” wherever that is.

C) Keyboard map to use:  Based on your language, American English is the default.

D) Detect and Mount CD Rom:  I don’t have one

E) Load installer components from CD:  Well, I have a stick but it works just the same

F) Configure the Network:  Choose your preferred connection.  Ethernet or Wireless

G) Configure the Network: Select your router if on Wireless, Select protocol (WPA) and give it your wireless passphrase.  It will connect to the internet and configure the network so it can grab software as needed.

H) Host Name.  Name the computer something meaningful like “server” or “debian”.  I always choose a moose related name like moose or rudolph, it doesn’t have to be anything serious.

I) Domain Name: For a home network, this is not needed, leave it blank.

J) Set Up Users and Passwords: Since they changed the way they use Root on debian with this version, leave the root password blank.  You can add the password later once you are up and running from a command prompt.

K) User account:  This will be your main user name.  Typically your own name, however debian will accept anything.  This will also be the only user that will be able to gain Root access because we purposely skipped the last step.  For this install I will be using bill.

L) User Name:  This will be who this person logs in at the login prompt.  Again, you can use anything.  I always have matched it up with the User Account name, so I will use bill again.

M) User Password:  The password you use here will be your user password plus your Root password.  Choose wisely, grasshopper, and do not forget.

N) Configure the Clock: Time Zone (Eastern for me)

O) Partition Disks: You can select “Guided – Use Entire Disk” and not have to calculate anything.  It is a little lazy, and there are other ways to configure it, however this is a basic tutorial and it saves me having to do some math.

P) Partition Disks – Select disk to partition:  You will be presented with a list of drives that are connected to the computer at this moment.  This will include the existing internal hard drive, your USB stick, and any other chips or disks that are connected.  My main hard drive came up called SCSI1.

Q) Partition Disks – Partitioning Scheme: It used to be that every separate tree had to have its own partition.  Then they realized “Math is hard, Barbie” and allowed you to select “All files in one partition”.  I do that.

R) Finish partitioning and write changes to disk:  Like the man says, this will configure the disk via a program behind the scenes and make the changes needed to install to the hard drive.

S) Write the changes to disks: Select Yes to make the changes live.  It will format your hard drive now to the partition scheme you selected before.

Following all that, it will install the system.  Get yourself a drink, it only takes me no more than 20 minutes, on a bad day.  Your mileage may vary.

T) Configure Package Manager: Use a Network Mirror – Yes.  Makes your life easier.

U) Configure Package Manager: Debian Archive Mirror Country: Select your country, United States was my default.

V) Configure the package manager – Debian archive mirror: Select a mirror that “looks” like it is close to where you are.  You can change it within the operating system later.

W) Configure the package manager – Proxy information: if you do not use one, leave blank.  I don’t so if you do, you’ll have to figure it out at this point.

X) Install the GRUB Boot loader on a hard disk: Select yes.  It will allow you to boot from your hard drive.

Y) Install the GRUB boot loader on a hard disk: Select the disk that you will be using to boot from, and that you installed to.  Typically this will be called /dev/sda and be the first disk on the list.


Z) Finish the installation:  You are done.  Remove your USB Stick or your DVD and hit enter.  The next thing you see will be the Debian boot sequence

This trip through the alphabet is brought to you by the number 6 and the letters debian.  Happy computing.  You now have a happily installed computer.

Relax!  And watch the boot process.  Or sip your drink.

When you finally get to the operating system, on XFCE Select “Use Default Configuration” for your desktop.  If you don’t then you have to hand configure everything instead of using the Default as a starting place.

How to get Tap To Click back on your Debian 9 XFCE Linux install

We are at an early point in Debian 9, and therefore many of the “downstream” distributions in Linux-Land these days.

Debian just made Debian 9, Stretch, the Stable version.  It also came out with an upgrade to 9.1 a couple days ago.

Since my own laptop was a Debian 9.0 install, I had a problem.  The track pad no longer did a “tap to click”.  It was there in the earlier versions, and removed in a Debian 9.0 install.  They migrated to libinst.  It promises to be new and shiny and do many new things but most of these things are in the future – or so my lack of Tap To Click would show.

I don’t use many of the more complex mouse options with my laptop.  It’s a non touch screen, Lenovo Thinkpad T530.  I heavily use Tap to Click so I want it back.  My other laptop, a Lenovo Thinkpad Yoga S1 had the same problem.  After a lot of research, this was shown to be a design decision.  Debian is my go-to operating system distribution due to the absolute depth of software and documentation out there.

So I set about to “fix it”.

DISCLAIMER:  I was able to do so on two computers but with some thrashing around.  I will give here the information that I have, but that thrash may make it less solid than my usual “cook book recipe” guarantee of any technical articles that I write.

Give it a shot.  If it works, let me know if you did anything different and I’ll mention it here.

Background – the documentation for Debian 9, Stretch, is still incomplete.  The files that I created had to be placed in Xsession.d and the directories that Debian gave were either missing or empty for me.  What they have is correct for the earlier versions and the docs need to be proofread.

Or I went crosseyed and got the wrong damn directory…

Since this blog is a place I put documentation for my own uses (Linux as well as recipes and photography), I’ll put it here.  I’d rather not have the heat of an official inquiry on me since I live in Florida and it is quite hot enough as it is.

First:  Create a 50-synaptic.conf – the file should probably not be there on a “clean install”

1) edit /etc/X11/Xsession.d/50-synaptics.conf

2) at the top merge (Copy and Paste) in the following lines:
Section “InputClass”
        Identifier  “touchpad catchall”
        Driver  “synaptics”
        MatchIsTouchpad “on”
MatchDevicePath “/dev/input/event*”
        Option  “TapButton1”  “1”
        Option  “TapButton2”  “2”
        Option  “TapButton3”  “3”
# This option is recommend on all Linux systems using evdev, but cannot be
# enabled by default. See the following link for details:
#       MatchDevicePath “/dev/input/event*”
EndSection
Second, copy that file to /usr/share/X11/xorg.conf.d/50-synaptics.conf
Third open terminal and sign into root to install a package:
apt install xserver-xorg-input-synaptics
Fourth: reboot.
On return, you should have tap to click working.  Entering “synclient TapButton1=1” on a command line should give you information for further research.
You may diagnose what the touchpad is doing by running as root “synclient”.
Entering “synclient TapButton1=1” on a command line should give you information for further research.
Further options such as multitouch, double finger tap for scrolling, and coast speeds and so forth are described in detail in the Debian Wiki Synaptics touch pad page at https://wiki.debian.org/SynapticsTouchpad

Migrating To A New Linux Computer With A Manifest

With Windows, you buy a new machine.  You copy a few things off the old one that you know are most important.  You make a token effort to re-create your old environment.  Then something Microsoft did gets in the way or you can’t find your original discs and you just keep it in the closet because you are afraid you will have lost all your data.  Because that’s what your buddy did down the block.

I’ve been told anyway.  I’ve also been told that most people have spare Windows computers that are taking up space.

Mac people can use a backup from Time Machine.  I’ve actually done that, and it is pretty slick.

I ran into a very different problem  My backups were perfect clones of the original.  But my original was “broke”.

Note:  This migration process is SO very easy that it takes about a half hour of actual hands on keyboard “work” and about 3 hours of processing time.  I have done this a couple times in a short span of time and am now getting “Creative” with the process.

Narrative:

Once upon a time, I installed Linux, and it was good…

Actually I installed Debian.  I figured that if there are so very many distributions of Linux that were forked from Debian, that Debian itself was safest.

I think I was right, no proof, just my opinion.  I have done my distribution hopping and had a machine in 1995 that was still being used in 2010 with CentOS 4.  Still stable, I just had much better hardware by then than my old Panasonic Omnibook with a Pentium 3 chip in it.  Yes, a 15 year long stretch with a computer is a long time, and I was the third owner of the machine.  It was my “pet”

I ran Debian 7 along side my Windows machines, and slowly found myself using Linux more than Windows.  I still use windows today, Windows 8.1 specificially, and I have an XP virtual machine With The Embedded Patch so I can get windows updates, but I don’t think I have run that within a month.

The only thing I use Windows for now is Photoshop, and really there are Linux programs like Gimp and Inkscape that will do what I need.

My original install was in my Dell.  Seven years ago in 2010.  First generation i7.  Dell Precision M4500. Blasted thing was built like a tank.  It loved, and once again loves, Linux.  I lived there for a year or more.  Then I was given upgrades, a couple times.  The original install went from machine 1 through 4.  Along the way Debian got upgraded to Debian 8, then recently 9 although I joined 9 back when it was “Testing”.

You see with Linux, you can clone the hard drive, take the clone, plug it into a new machine, and it just may work.  All you need is a USB caddy for the destination drive and as long as your drive names line up it works.  dd if=/dev/sda of=/dev/sdb bs=4M conv=noerror,sync

For the most part it did but there were weird video affects and strange hibernate and resume problems. This cropped up as a result of taking machine 1’s operating system and making it stable in machine 2 and 3 and later 4.

The actual process:

So here I am, creating from scratch Son Of Original Install. Debian 9 with XFCE4.  Oh, and a lot of extra “baggage” that I don’t need but it is easier that way.

I decided that I would create a list of programs that I could reinstall on the New Machine and see how it works.  Also this is done with me in XFCE4.  We Linux People are if nothing else, flexible.  If it is not where I said it is, poke around a bit.

Step 1.  New Machine, is a Thinkpad T530, and gets a clean “Bare Metal” install of Debian 9. I ended up doing it a couple times, and so far the only weirdness is that it insists that I do “sudo su” if I want my terminal session to be and remain root.  They also renamed the network devices that have been used for decades.  So when I get to the network tweaks that I will have to do, I may have to edit a configuration file. Most likely samba.conf.

Success.  I’m typing this from that machine now.

Step 2.  Create Manifest and install it from Synaptic. (Menu, System, Synaptic)

Step 2a.  On Original, open Synaptic.  After giving it the password for Root create a manifest by clicking “File, Save Markings As” and ticking a box at the bottom of the window that says “Save full state, not only changes”.

Synaptic created the file with everything and in next step, it will place everything where I need it.  Yes, it will add a lot of software I don’t really need, but with Synaptic and Linux, I can purge all that stuff with a simple “apt purge” and it will remove it all, completely.  Put it on a chip or USB stick and place it on the new machine.

You can edit the file and delete anything out that you know won’t be needed, but you will have to trust Synaptic to realize what you’re trying to do.  Best if you did the removal in the next step.

Step 2b.  On New install on the new machine, open Synaptic and select “File, Read Markings”.  Tell it where that file is.  It will read it in and select all your “markings” from the manifest and instruct Synaptic to later install.

I did that in bed.  It was 8GB worth of upgrades on a replacement for my old 7 year lived in install.

Here is where I am second guessing and should have removed the other programs and window managers that I don’t use.  I like XFCE4, it’s light, fast, and configurable.  Others prefer KDE or Gnome.  I have them all installed.  Why not, it’s a seven year old install.  If you remove it before telling Synaptic to update, Synaptic will get rid of the chaff along with it.  I didn’t want to, I wanted “What I Had On The Old Install”.

Step 3.  Bring over my home directory.  I cloned the Original install on a backup drive.  I took that drive and plugged it into an external case.  Plugged that into the USB port. It is copying.

Step 4.  Live with it.

I have to go with this new install for a while spotting problems.  And I haven’t gone back to the old machine since.

Step 4a) The first one was I had to be able to play DVDs.

Change, as root, the /etc/apt/sources.list file by editing it and adding four lines:

#2017-07-08 to add libdvdcss

deb http://download.videolan.org/pub/debian/stable/ /
deb-src http://download.videolan.org/pub/debian/stable/ /

Then install as root by “apt install libdvdcss2”.

VLC worked by playing Futurama in Spanish.  Leela is a babe.

Step 4b) network shares on windows are not yet accessable.

SAMBA was installed on Original, and it was happy.  It took a lot of twiddling to get that there.  Luckily I could copy over and merge it into my bare bones samba.conf file.  I saved the new one as installed, then copied the one over from Original, then restarted.

Fixed the access to my network shares.  It did not fix the share I had on the new machine.  I’ll work on that.

Step 5) Conclusion is that this process works.

Worked.  I’m on day 6 of all of this.

Two problems cropped up:

1) The network share on my “new” machine still hasn’t been fixed but I will deal with that later

2) Flash does not work.  Flash, as a platform, is dying. The only place it irks me is on www.imgur.com when I run across a short video to play.  I’ll look into that at my leisure.

Step 6) Epilogue:

Furthermore…  I got bored and did it again with another machine.  I had it working once the updates happened.  It’s a first generation Thinkpad Yoga S1 and has its own problem.

That’s the thing, there will always be quirks.  Be prepared.  They happen because the new computer has different hardware than the original one.  You may need drivers, and you may need to remove software.

After all you still have your old machine and its backup, so you can go back if you want.  This “migration” is completely safe to your original data.

Docker on Debian Linux – Why and The Install of it All

If you want just the instructions, Skip to the break.  This is here basically so that I can do this again later.

The setting:

For almost all of what I do, I run Linux.  Specifically Debian Linux.

It runs much faster, has most of the same programs you’re used to on a Mac or a Windows PC, and is about as stable as an operating system can be. It can run some Windows programs in Emulation (WINE) but that’s not the point. I’ve got what I need if I stay within Linux, natively.

Some Debian Linux computers have “uptime”, time since they were last restarted, in years – not days or weeks.

I update things when I want.  I make things how I want them.  I change things how I want.  If there is one thing about Linux that Mac and Windows users don’t get to do is customize things the way that they want.

The backstory:

My blog resides in two places.  On www.ramblingmoose.com and on a WordPress hosted site at ramblingmoose.wordpress.com as a backup.

I really don’t care for how my WordPress site looks, so I want to change it.  Being someone with more years in IT Software Development Project Management than I care to admit to, I will do it “offline” and not on the live site.

Furthermore, I have a client in Los Angeles.  His website was developed on WordPress by me, and I have a backup.  I’m finished with the site, but I thought it might be “fun” to see if I could get it to work here on my own computer.

The reasoning:

My own main computer running Debian I am happy with.  Actually that is an understatement.  I don’t want to slow it down by loading up server software, a LAMP stack, and things to slow it all down.   I could create a VMWare or Virtual Box virtual computer and do the LAMP stack there, I’ve done that a couple times before, but running a full VM for something like this felt “overkill” and “heavyweight”.

What Docker Does:

Docker will allow me to share some of my computer by running a pared down version of Linux inside what they call a Container.  It is not a full virtual computer, so it should run faster, and since it is not a full computer it will not effect my apparent speed – in case I forget to “turn the damn thing off at night”.

The Goal:

Get a Docker Container up and running.  The container will have a web server and WordPress software running configured for my use.


 

Installing Docker:

This is adapted from the official Docker instructions found at this link.

This will get the base Docker software installed on a Debian system.  Your system should be “up to date”.  It should be running fairly current software.  As I am writing this March 2017, Docker will run on Debian 7, 8, and 9.  9 being “Stretch” or “Testing” at this point, 8 being “Jessie”, and 7 is “Wheezy”.

My own personal thought is that if you aren’t running at least “Jessie”, get yourself upgraded to current software.  After all, within a month or three of this writing, Stretch will become “Stable” and the official up to date current software.  It’s easy.  I started with Wheezy, migrated to Jessie, and am currently running Stretch.

Open a terminal session and sign in as root with “su”.

Add transport module to allow Docker to grab what it needs via HTTPS:

Add the Docker GPG key:

curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add –
Verify that the key ID is 9DC8 5822 9FC7 DD38 854A E2D8 8D81 803C 0EBF CD88.

apt-key fingerprint 0EBFCD88

pub   4096R/0EBFCD88 2017-02-22
Key fingerprint = 9DC8 5822 9FC7 DD38 854A  E2D8 8D81 803C 0EBF CD88
uid                  Docker Release (CE deb) <docker@docker.com>
sub   4096R/F273FCD8 2017-02-22

Add the Docker repository:

add-apt-repository \
“deb [arch=amd64] https://download.docker.com/linux/debian \
$(lsb_release -cs) \
stable”

Update your apt repository lists:

apt update

 Install Docker:

apt install docker-ce

 

At this point, Docker is installed.  They set up a container that you can run to verify it called Hello World.

docker run hello-world

Docker is installed.  The software will run from the command line, as root.  It can be configured to start automatically by entering in a command:  

systemctl enable docker

However I have not yet done it because I am not convinced I want to run this every time I boot.  I am being conservative with my system resources, but to be honest I have not noticed it slowing me down in the slightest.   Since I will only be running this intermittently,  I probably will not be running it.

Since this tends to be my own mental scratch pad, the way to disable docker at boot is:

systemctl disable docker

This and more tweaks to how it runs are found at Docker’s own help file for post install.

Conclusion:

Obviously, this is something that is incomplete.  I will be returning with more when I go to get my container started.  I need a Container with Debian, a LAMP Stack, and WordPress.  A ready made version of this exists, and I will try that first – ready made from Docker itself!

On the other hand, my own normal IT Project Management curiosity tells me that I need to make one on my own.  So I’ll work on that later.

First step, getting it installed, worked.  Next I will get onto that other stuff… later.

Cloning a Hard Drive With Linux

Yeah well calling it Linux means I most likely lost 97% of the market.


Windows people don’t realize that there is a painless way to get their windows computer to do some of this stuff – a Live Linux Distribution like Ubuntu.  If you get a live disc working, you can copy this shell into it, then follow the instructions.  It should work.

Mac people may even be able to run this natively.

Maybe.  Depends if PV is Mac Friendly, if not, convert the PV line to a copy of your choice.


A Live Linux can be “burned” to a USB stick or to a DVD and your computer can be booted from that.


And now you know!


But none the less…

What this is basically is my own shell.  I use this to completely back up my computer.  All the drive specifications are found and known, and do not change.

I run fdisk -l as root and use the information in there to edit the shell script to change things as needed.

This assumes that you know what your drive devices are, are willing to edit a shell script to make your own changes as is, then have an external USB hard drive slightly larger than your boot device.  My boot device is /dev/sda and most likely yours is as well.

This assumes that you have a second drive sitting in your chip reader.  If not, you can comment out the line that copies it to the hard drive.

This assumes that you have room enough to do everything.

I am doing this on Debian Linux, however the commands here are so very generic that you should be able to run this on most “full” distributions of Linux.  Debian, Ubuntu, Linux Mint, Centos, Fedora and the like come to mind.

Standard Internet Warranty – I make no warranties and it is at your own risk.  If you lose data, it is on you.  I take zero responsibilities for any miscoding or changing or whether a magic dragon comes out of the skies and takes you onward to valhalla.  Really.  None at all.

I will say that I ran this exact shell this morning and it worked for me.  You WILL have to change the file specifications to fit.   

Finally:

  • My boot drive is a 240gb SSD with about 120gb free.
  • My chip has about 12 gb worth of data on it.
  • Debian thinks that the chip is called “128GB” and it typically comes up in the file manager (thunar) on /media/bill/128 GB/

Prerequisites:

Installed versions of

How it runs:

  • This must be run as Root in Terminal.
  • This will pause after each step with an OK message in the Dialog box.
  • For me, the entire shell runs in about 2 hours on my i7 laptop with a USB 2.0 external hard drive.

First the shell in its entirety through to the end comment:

#! /bin/bash


#backup.sh from http://www.ramblingmoose.com

dialog –no-lines –title ‘Run This As Root’ –msgbox ‘This shell will backup SDA to SDB\nYou must click OK after each step so watch this.\nYour Disaster Recovery will thank you!’ 10 70

dialog –no-lines –sleep 3 –title “update your sources” –prgbox “apt-get -y update” 10 70
dialog –no-lines –sleep 3 –title “update your software” –prgbox “apt-get -y upgrade” 10 70
dialog –no-lines –sleep 3 –title “update your distribution” –prgbox “apt-get -y dist-upgrade” 10 70

arg1=”‘/media/bill/128 GB'”

dialog –title “copying the chip to the drive” –prgbox “cp -avr $arg1 /home/bill/128GB” 10 70

(pv -n -i 2 /dev/sda &gt; /dev/sdb) 2&gt;&amp;1 | dialog –title “Backup SDA to SDB” –gauge ‘Progress…’ 7 70

dialog –title ‘Message’ –msgbox ‘Cloning is done, click ok to clean up and end’ 5 70

dialog –no-lines –sleep 3 –title “Removing the copy of the chip” –prgbox “rm -r /home/bill/128GB” 10 70 

dialog –no-lines –sleep 3 –title “Synchronize your drives” –prgbox “sync” 10 70
#end backup.sh

To actually use that mess…

  • Copy the entire text and paste it into your favorite text editor.
  • Save the file with a “.sh” extension somewhere you will be able to get to it – in your path.
  • Change the mode to executable – chmod 0770 backup.sh
  • Change the owner to root.  You never want to use this as a regular user – chown root backup.sh
  • Change the group to root.  chgrp root backup.sh
  • Run the shell as root: sudo ./backup.sh

Now, each line in excruciating detail!

—- Run the programs using bash interpreter

#! /bin/bash

—- I’m signing my work here

#backup.sh from http://www.ramblingmoose.com

—- This puts up a message box

dialog –no-lines –title ‘Run This As Root’ –msgbox ‘This shell will backup SDA to SDB\nYou must click OK after each step so watch this.\nYour Disaster Recovery will thank you!’ 10 70

—- The next three steps gets your distribution to date.  Don’t want this, comment it out

dialog –no-lines –sleep 3 –title “update your sources” –prgbox “apt-get -y update” 10 70
dialog –no-lines –sleep 3 –title “update your software” –prgbox “apt-get -y upgrade” 10 70
dialog –no-lines –sleep 3 –title “update your distribution” –prgbox “apt-get -y dist-upgrade” 10 70

—- Store the directory that Linux mounts the chip to in “arg1”  If no chip to backup you can comment this.

arg1=”‘/media/bill/128 GB'”

—- Wrap the actual work of copying the chip out to a dialog box.  The flags “-avr” say copy the whole drive in $arg1 recursively to the destination.  If no chip to copy, comment this line.

dialog –title “copying the chip to the drive” –prgbox “cp -avr $arg1 /home/bill/128GB” 10 70

—- This line does the real work.  Now that you copied your chip out to the hard drive, clone the actual hard drive.  The flags on pv tell it to report to stdout the percentage of work done so that dialog can show a pretty gauge.  Ahh, so pretty!

(pv -n -i 2 /dev/sda &gt; /dev/sdb) 2&gt;&amp;1 | dialog –title “Backup SDA to SDB” –gauge ‘Progress…’ 7 70

—- Copy is done, it is time to clean up message

dialog –title ‘Message’ –msgbox ‘Cloning is done, click ok to clean up and end’ 5 70

—- remove the data that you copied from the chip from the hard drive to be neat. if no chip, comment this out.

dialog –no-lines –sleep 3 –title “Removing the copy of the chip” –prgbox “rm -r /home/bill/128GB” 10 70 

—- Your work is done, make sure you flush your cache by doing a “sync”.

dialog –no-lines –sleep 3 –title “Synchronize your drives” –prgbox “sync” 10 70  

#end backup.sh

Netbook Server – Sharing An External Hard Drive In Linux

So if you have followed my instructions, you now have a:

Computer that runs Debian Linux
http://www.ramblingmoose.com/2016/02/the-netbook-server-installing-debian-or.html
Computer that you can look into using Remote Desktop
http://www.ramblingmoose.com/2016/02/the-netbook-server-you-need-to-be-able.html
Computer that you can share part of the local hard drive
http://www.ramblingmoose.com/2016/02/the-netbook-server-how-to-actually.html
Congratulations.  You now have a file server!

If you followed those directions, it also installed a bunch of other programs that will let you do other things.  I noticed that something called “CUPS” was installed, and that will let you plug a printer into the same machine and act as a “Print Server” or a “Network Printer” – if you can find the instructions on how to configure it.

Debian and Raspbian both come with enough that you could use that machine as your one and only daily driver computer.  The browser is called “Iceweasel” and is Firefox, rebranded.  You have Libre Office to write letters, work with spreadsheets, and make presentations that are all compatible with Microsoft Office.

Yes, it really is, I use it every day.  No, you don’t have to pay for it.  Ever.

There are more apps, and I would suggest looking into some of the software that is out there, all free.  If you start “synaptic” from your terminal as root or “sudo synaptic &” you will find so much free software that your mind will fog up and get tired before you find everything you want.

But that all is just the preamble to this discussion.  You came here to share an external drive.  This is like any other shared drive on the network, you have to have it plugged into the server (USB Port on your netbook), you have to tell the computer where it is, and you have to tell it how it is to be shared.

Remember, I am trying to write this for a Windows audience so I’ll go as basic as I can.  You Windows folks are in a new world, and you will want to have this go well.  If you are a Linux expert or even intermediate, you may find this needlessly wordy.   Not to worry, you’ll be right.

One Step At A Time.  Divide and Conquer.

First step – Make sure you can read the drive from Linux.


Before you get anywhere, start the computer.  Log in.  Get to your desktop.  Then plug in the drive.

Start your terminal session by clicking on the (start) “Applications Menu”, then click on Terminal.  Sign in as root by entering “su” and your root password.  You will eventually need this

Now, launch the file manager by clicking on the (start) “Applications Menu”, then click on “File Manager”.

In the left pane of the file manager you will see Devices, Places and Network.  In “Places” your external drive will come up with a little eject arrow to the right of it.  Click on the icon for the drive.  A little wait icon will start to rotate.  When it is through it will do one of two things:

Success is if you are dropped into a view of whatever files are on the disc.  It means that all the drivers are in place.  Most likely this drive is something called “vfat” or “fat32”.  Remember this for later.

Failure is if you get a big ugly warning message up.  That means that you don’t have the drivers for the format that the drive has on it.  Most likely you will have to install the set of drivers called “ntfs-3g”.  This would be where your external is a really big drive and you did it to make things faster.  To install that do the following steps:

  1. apt-get update
  2. apt-get upgrade
  3. apt-get install ntfs-3g
  4. shut down the server
  5. unplug the drive (It isn’t shared yet and you don’t want to wait for the computer to release it)
  6. start the server
  7. and plug in your drive when you have logged back in to the desktop, terminal, and file manager.

No matter what, at this point, you should be able to read your external drive.

You also need information.  When you worked with the server software “samba” you created a user and a password, and you will need that later.
 

Next step – finding where Linux thinks that drive actually is.

Here is where Linux people will be saying “gparted“.  If you know how, go for it, this is the slower but less risky method.

To determine what is plugged into your machine type into the terminal:

  • dmesg | tail -30

Linux keeps a log of whatever is important to the system.  Since you “just” plugged that external drive into the computer, the last thing on that very long stream of text will be what was reported when the computer detected the hardware.  The “tail” bit will tell terminal to just show the last 30 lines of what are in the display of messages (dmesg).

The clue there are the lines that say “usb 1-2” and “sdb”.  When I plugged in the drive, it said “new high-speed USB device number 2”.  So what we’re going to tell the system is that the drive is sitting on a device called sdb.  The partition we will be using will be the first one, so it is officially “/dev/sdb1”.  In windows, it would come up as your D drive if there is no DVD/CD drive present, E Drive if it is, this is the same thing.

Since my stick is formatted to be removable on Windows, it is a format that Linux calls “vfat“.    My big 4 TB drive is formatted NTFS, so I would have to mount it as “ntfs-3g

Create a place to store the data in.  In my case, it is “/home/bill/external“.  You should change “bill” to the name of your user that you logged into when you started this exercise.  To make the directory, open terminal again as a regular user and enter this command:

  • mkdir /home/bill/external
  • chmod 0770 /home/bill/external

You just created the directory and set it up so that you and root can use it.

There is one file that you need to edit in Terminal with the following command:

  • nano /etc/fstab

This file tells linux where all of your disc drives sit, so be careful and don’t delete anything.  You will be adding a line, as below:

  • /dev/sdb1 /home/bill/external vfat defaults 0 0″

That says – put the external drive’s first partition “in” the /home/bill/external directory.  It also says that it is “vfat” format so change that if it is an ntfs-3g format.  The defaults are lengthy and you can go into them in great detail on the Wikipedia Article.

If you wanted to go further and add multiple partitions for other people, you could do it in /etc/fstab by adding multiple entries.

Once you restart the computer, you should be able to find the drive on Windows, and you are on your way.  Just find the drive in Windows File Manager, enter in your login from Linux, and you’re good to go.

One final wrinkle

What this does is to “bind” the external hard drive or memory stick to the server.  It is now set to automatically mount and share the drive whenever the power comes back on.  If you do not have a drive plugged in, Linux will boot, but put you into a terminal session as root into what is called “Single User Mode”.  You can do the following edit at that point with the commands below.

To remove the hard drive so that the server is no longer looking for the drive at boot, in terminal as root:

  • nano /etc/fstab
  • find the line with the external drive and enter a # as the first character in the line
  • save the file and restart the computer

This now turns your server into a machine that only serves the local hard drive.