Dropbox backup script

When I was working on my PhD I used Dropbox as extra insurance against accidentally deleting all of my thesis files, or forgetting to take the most recent copy with me when I went to work away from my desk. Dropbox can be a bit confusing to use, though, and if you are not paying attention you can end up deleting large numbers of files from your hard drive. I managed to delete my entire PhD directory when I first started using Dropbox! Fortunately if you notice it soon enough you can generally recover them, but it is a pain.

The other issue is that I don’t want to keep the files I want to sync in the Dropbox folder itself. You can use symlinks but I am always a bit paranoid about accidentally deleting the source directory. For this reason, I have finally gotten around to creating a script that copies certain folders and files into my Dropbox folder, rather than using symlinks. With this method Dropbox acts more as a true backup, because the files that are synced are copies.

I just added the script to my /etc/cron.daily folder. I had never messed around with this before but it makes it very easy to run programs or scripts automatically. Anacron only runs once a day, however, so when I created my automatic email backup script during the most intense PhD writing period, I added it to cron instead, so that it would run several times per day.

Here is the script I have just created. It took some fiddling to get everything in the rsync command the way I wanted it, but this works now. I have some symlinks to my main BibTeX file on my main system, but this script uses the -L option to just copy that as the file itself wherever the symlinks are. It also makes a copy of this file in the parent directory, just as a convenience.

Note that the em-dash (–, created when you have two – characters) should be written as two dashes in the script.

#!/bin/bash
# copy research files to Dropbox directory
RESEARCH=/home/me/Documents/Research
DROPBOX=/home/me/Documents/Dropbox/Private
#
# rsync options used:
#-v, --verbose increase verbosity
#-r, --recursive recurse into directories
#-u, --update skip files that are newer on the receiver
#-L, --copy-links transform symlink into referent file/dir
#-p, --perms preserve permissions
#-t, --times preserve modification times
#-z, --compress compress file data during the transfer
#
rsync -vruLptz --include=Subdirectory1/ --include=Subdirectory3/ --include=Subdirectory5/ --exclude=/* /$RESEARCH/ /$DROPBOX/
cp $RESEARCH/References/BibTeX_Database.bib $DROPBOX/BibTeX_Database.bib
cp $RESEARCH/References/BibTeX_Database.bib.bak $DROPBOX/BibTeX_Database.bib.bak

An OS install and XFCE thoughts plus an annoyance, solved

So I upgraded to Fedora 17 a while ago, after doggedly holding on to my outdated F13 installation because I had everything customized *just so*. I’d heard plenty about how terrible Gnome 3 was and I wasn’t all that keen to upgrade. When I eventually did I didn’t think that the new Gnome was all that bad, but after using it for a couple of months I decided I didn’t actually like it very much and started trying out alternatives. Sometime in the near future I am going to upgrade again to Fedora 18 and I think this time I will just pick one desktop manager from the start rather than installing all of them. What an idea…!

Right now I’m using XFCE, in part because I decided to try it on my netbook and I thought it would be nice to use a similar desktop on both computers. The netbook was given to me by a friend of DH’s because it was not working properly and he got fed up with trying to fix it. Not only was this very generous in itself, but I really prefer this one to DH’s Acer because it has a large hard drive, so I can use it as a backup device. After much fiddling, including tinkering with the hardware, I decided that the memory must be faulty even though all the tests I ran said it was not. So I swapped it for an old 512MB card leftover from upgrading my main computer, which seems to have done the trick. But since that’s half the memory it had before I wanted a relatively low-memory OS. I tried DSL, Puppy, and antiX, but they are all a bit fiddly and for some reason I just could not get antiX to mount USB drives. I really want to like Puppy but installing extra software is not that straightforward and I never did find a LaTeX system that worked well before I got fed up. All this is a roundabout way of saying I ended up installing Xubuntu on it and it works pretty well so far.

One thing that irrationally annoyed me about Gnome 3 is that the panel is not easily editable and you have to install a bunch of extensions to get the silly features I grew accustomed to, like the weather and timer applets. I still don’t have the timer working on XFCE (there is a package to make Gnome applets work on XFCE but it’s not in the repository and I’ve been too lazy to mess with it yet), but the weather applet works. The menu is still not as easily customized as the old Gnome 2 menu, although some of the confusion on my main system might be due to the fact that I still have Gnome 3, KDE, LXDE, MATE, and XFCE installed. On the netbook it is nice and simple, however, and everything works reasonably fast. I did do some fiddling with swappiness, which seems to help it run faster, although it unsurprisingly lags a bit when I have four or five programs going at once. Although I use Kile on my main system I didn’t want to install all the KDE dependencies on the netbook so for now I am using an editor called TeXStudio, which is a fork of TeXmaker.

Now, for the annoyance. XFCE in its wisdom has decided that alt+f1 should call the system help, rather than the main menu. Although there is a tab in the keyboard settings that would appear to give you a way to alter these, it did not seem to actually work. I ended up deleting the shortcuts I wanted and adding them back, which then finally brought up the dialog for assigning a new shortcut. I still haven’t figured out how to turn off tap-to-click on the netbook, though. Various solutions I have tried don’t actually seem to work, and since it is not hugely important I have set it aside for now.

Installing cb2bib on Fedora 17

Clearly, I have been neglecting my academic life, to the extent that I only realized yesterday that I didn’t re-install my favorite program (cb2bib) for processing PDF articles and keeping my BibTeX database up-to-date when I upgraded my system several months ago. Unfortunately there isn’t a version in the Fedora repository so I ended up building it from source, using the tarball instructions.

This should not actually be difficult. While I don’t know how to code these things myself, I am perfectly capable of typing ./configure, make, and make install. Easy, right? It is if you have the right dependencies installed…

After much fiddling around and many rounds of yum search, I finally managed to install everything I needed. I thought I already had everything but it seems that I had not installed the appropriate versions of everything, so I ended up adding cmake, gcc-c++, qt-devel, and lzo-devel. I already had Qt 4.4.0 installed but the scripts were still looking for the Qt 3 folder, so I updated my PATH* to Qt 4 folder (and deleted the reference to Qt 3). Rather than ./configure –prefix /usr/local I found that ./configure on its own completed without errors. Finally, after make and sudo make install, I am in business again!

Every day I use Linux, I learn something new. I suppose that is a good thing, but it does mean I spend a lot of time searching man files and googling for answers!

*I changed the PATH following these instructions. Rather than fiddling around with using sed to delete the Qt 3 reference, I just printed the existing settings using echo $PATH, then copied and pasted that into the PATH= command (deleting the unwanted value, of course!).

Adventures in LaTeX formatting, part 3: bibliography

The most complicated issue was formatting the bibliography and putting it in the right place. Ideally, since her thesis was of the “separate papers with an introduction and conclusion” model, she wanted to have each paper as a chapter with a separate reference section. In theory, this is fairly simple to do using the bibunits package, for example:

\usepackage[sectionbib]{bibunits}
\bibliographyunit[\chapter]
\defaultbibliography{MyBib}
\defaultbibliographystyle{plainnat}

In this case, she wanted the references to start on their own page and be single spaced, rather than 1.5 spaced like the rest of the thesis text. This can be accomplished with the following at the end of each chapter:

\clearpage
\begin{singlespace}
\putbib[NewBib]
\end{singlespace}

The only problem with this is that running BibTeX normally only creates .aux, .blg, and .bbl files for the first chapter. LaTeX then cannot find all of the information so the other chapters are filled with ?? instead of citations, and the references sections aren’t created. I tried various methods of getting BibTeX to run multiple times, creating separate files for each chapter, but there were two further issues. First, the files get overwritten on each run. I solved this by telling BibTeX what to call each file:

\makeatletter
\renewcommand{\@bibunitname}{\jobname.\the\@bibunitauxcnt}
\renewcommand{\@bibunitname}{bu.\the\@bibunitauxcnt}
\makeatother

Second, if you are using a GUI-based editor program to write and compile your text — as most people who are not hardcore geeks would do — then only the standard LaTeX – BibTeX – BibTeX – LaTeX sequence of commands is run, and only on the first chapter. I tried several things involving \write18 (including -enable-write18 in the program settings) to try to get BibTeX to run on each chapter, but none of them worked from within the editor program due to the security settings. Since my friend uses a Mac and has never used the command line, asking her to dig into the guts of her system and change important security settings was a non-starter, especially since we are in different countries and I use Linux. But it is quite easy to get everything to work properly from the command line.

To get around her lack of familiarity with the terminal I made a script that bundles up all of the commands and runs them in the right order, and also converts the resulting .dvi file to a .pdf. It would have been nicer to use pdflatex so that the end file was nicely indexed with clickable bookmarks, but this would have involved messing around with converting graphics formats. Since this whole business was conducted via email it would have been a serious pain for her to email me every file, and since she was too stressed and panicked to learn how to do it on a Mac it seemed best to just get a functional document with fewer bells and whistles.

The script itself is pretty simple:

#!/bin/bash
echo "Please enter a LaTeX file name (WITHOUT .tex):"
read file
latex $file
latex $file
for bibfile in *.aux;
do let j+=1; mv $i bu$j.aux;
bibtex $bibfile;
bibtex $bibfile;
done;
latex $file
latex $file
dvipdfm $file.dvi

Since I don’t have access to a Mac and wasn’t sure how to create a nice little point-and-click program, she still had to run this in a terminal. I added the prompt so that it was very clear what to do. Since Mac is UNIX-based, however, common utilities like bash should work. Unfortunately this meant she still had to change permissions and do some other command-line hocus pocus. I cobbled together these instructions for running the script:

How to run this in Mac OS X:
- copy the file Script.sh to your LaTeX directory (e.g., /Users/Me/MyFiles)
- get a shell prompt (command is Run Terminal). You can find Terminal in the Utilities folder inside your Applications folder.
- in Terminal, type: cd /Users/Me/MyFiles
- in Terminal, type: chmod u+x Script.sh
- in Terminal, type: ./Script.sh
You will be prompted to enter your file name, just type “myfile” without the .tex extension
Hit enter, and after a minute or so it should spit out a .dvi and a .pdf!

In the end she was not comfortable running the script, so she changed to format to a single references section at the end of the text. Although this was the easiest solution, it led to yet another complication. As she is an international student, she included summaries of her thesis in both English and another language. According to the specifications these had to come after the references section. However, by default the references section in the book class has the heading “REFERENCES”, which is repeated in the page header until the end of the text — even if the appendix environment is invoked, or a new part is started. The most expedient way around this was to simply redefine the command by copying the relevant section in the standard book.cls file, changing the options, turning it into a macro by enclosing it in the \makeatletter and \makeatother commands, and pasting it into the preamble. Like so:

% COMMANDS TO REMOVE "REFERENCES" HEADER
\makeatletter
\renewenvironment{thebibliography}[1]
{\chapter*{\bibname}
\addcontentsline{toc}{chapter}{\bibname}
% \@mkboth{\MakeUppercase\bibname}{\MakeUppercase\bibname}% % this is the default option in book.cls
\list{\@biblabel{\@arabic\c@enumiv}}%
{\settowidth\labelwidth{\@biblabel{#1}}%
\leftmargin\labelwidth
\advance\leftmargin\labelsep
\@openbib@code
\usecounter{enumiv}%
\let\p@enumiv\@empty
\renewcommand\theenumiv{\@arabic\c@enumiv}}%
\sloppy
\clubpenalty4000
\@clubpenalty \clubpenalty
\widowpenalty4000%
\sfcode`\.\@m}
{\def\@noitemerr
{\@latex@warning{Empty `thebibliography' environment}}%
\endlist}
\makeatother

With this version, the standard bibliography command was sufficient to produce the references section, again in a singlespace environment:

\begin{singlespace}
\bibliographystyle{plainnat}
\bibliography{MyBib}
\end{singlespace}

And finally, in order to add the summaries as separate, non-numbered chapters but which also appear in the table of contents, I used these commands:

\chapter*{English Summary}
\addcontentsline{toc}{chapter}{English Summary}

\chapter*{Other Language Summary}
\addcontentsline{toc}{chapter}{Other Language Summary}


Adventures in LaTeX formatting, part 2: longtables

The next problem my friend had was with longtables. This required a bit more fiddling than I would have liked, because of the content of the data in the tables. They were originally broken up into smaller tables that dealt with particular themes, but her supervisor wanted them to be combined.

The two long tables had to be less than three pages, and had a column of text that needed to wrap to more than one line on some rows. Each item was presented in two rows: the first with a value, and the second with a standard error statistic, which was enclosed in parentheses. Some values had asterisks and minus signs, and they all had at least two decimal points. In order to line everything up in a way that was readable and fit within the page limits, I had to define some new column types and use multirow in a few places.

I used dcolumn to create a column type that was aligned on the decimal point, rather than creating a separate column for the numbers to the left and right of the decimal point. The latter approach is the one that is usually recommended, but to me it seems prone to mix-ups. Also, with dcolumn you can define how much space to put on either side of the decimal, making it easier to accommodate the parentheses and asterisks in the current example.

The d columns were created like so:
\usepackage{dcolumn} %align numbers by decimal point
\newcolumntype{d}[1]{D{.}{.}{#1}} % D{sep.tex}{sep.dvi}{number.format}

I also wanted the text in these columns to wrap to the next line and to be left-aligned, rather than justified. The option “raggedright” in the array package doesn’t work properly in a table environment, apparently due to a conflict in the way that “\\” is interpreted, so I used ragged2e to create column type P:
\usepackage{ragged2e}
\newcolumntype{P}[1]{>{\RaggedRight}p{#1}}

These two column types are invoked in the longtable environment like so:
\begin{longtable}{P{5.5cm}*4{d{2.4}}}

In this case the table had 5 columns altogether. The first one uses the P type and specifies the width in order to make the text wrap within the cell. The other 4 use the d type, and specify that there can be two characters before and four characters after the decimal point.

I tried various ways of applying a hanging indent to the table cells when the lines wrapped, but none of the options I tried for making this a universal command seemed to work. So in the end I had to adjust the longer rows with complicated-looking commands like this, which makes the cell span two rows *and* embeds a minipage in the table cell so that I could then apply the hanging indent to the text:

\multirow{2}{*}[5pt]{
\begin{minipage}{5.5cm}\everypar{\hangindent0.5cm \hangafter1}
LONG TABLE ROW TEXT
\end{minipage} }

There is surely a better way to do this, but I didn’t find it in the time I had available. Eventually I will figure it out for my own satisfaction. In an ordinary tabular environment you can use something like:

\begin{tabular}{>{\everypar{\hangindent0.5cm \hangafter1}}p{5.5cm}ccr}

but it doesn’t work in a longtable.

You can specify the text that appears at the end and beginning of each page in a longtable, including the formatting. There are separate options for the first page header, last page footer, and the pages in between. Since the column headings were in the form “Column 1″, they came out aligned to the left and, for reasons that I didn’t have time to figure out, in italics. The easy way around this was to wrap each one in a multicolumn command, which overrides the italics and allows the text to be centered without centering the entire column. So we end up with options like this:

\hline \hline & \multicolumn{1}{c}{Column 1} & \multicolumn{1}{c}{Column 2} & \multicolumn{1}{c}{Column 3} & \multicolumn{1}{c}{Column 4} \\ \hline \endfirsthead

\multicolumn{5}{c}{\tablename\ \thetable{} -- continued from previous page} \\
\hline \hline & \multicolumn{1}{c}{Column 1} & \multicolumn{1}{c}{Column 2} & \multicolumn{1}{c}{Column 3} & \multicolumn{1}{c}{Column 4} \\ \hline \endhead

\hline \multicolumn{5}{r}{\footnotesize Continued on next page} \\ \hline \endfoot

\hline \hline \endlastfoot

I think I have mentioned this before, but my secret trick for making tables in LaTex, especially when there are lots of complicated nested commands like this, is to do the actual work in a spreadsheet. If you are starting with a complete table in LaTeX, it helps to first add another character in front of & and \\ with find and replace. I often use |, but anything that isn’t meaningful in your document will be fine. Copy and paste the table into your spreadsheet and separate the text into columns using your character. This makes it much easier to repeat complex commands across rows while keeping everything straight. When you are finished, simply copy and paste the spreadsheet back into a text editor. This will usually introduce extraneous tab spaces, but those are easily removed using find and replace.

Adventures in LaTeX formatting, part 1

A few weeks ago I helped a friend make some formatting changes to her thesis, which had been requested at the last minute by her supervisor. The changes were seemingly pretty simple — make first-level headings all caps, put the references before the appendices, tweak a few tables — but there were complications. Even though I love LaTeX for the simplicity of changing things like this with a few lines of code in the preamble, it can still be tricky if you don’t know what you are doing. But with a bit of trial and error (and lots of googling) I managed to get everything to work for her. I am documenting my efforts here so that when I inevitably forget what I did and want to apply it to my own work, I can find it again.

There were several issues, which would make for an unreadably long blog post, so I am separating them into three posts. The least complicated of these was changing the format of the chapter and section headings to follow her supervisor’s specifications. This was actually pretty easy to do. I used the titlesec package to override the default settings in the book class, as follows:

\usepackage{titlesec}
\titleformat{\section}[block]{\normalfont\normalsize\bfseries\filcenter}{\thesection}{1em}\MakeUppercase{}
\titlespacing*{\section}{0pt}{3.25ex plus 1ex minus .2ex}{0.5em}

\titleformat{\subsection}[hang]{\normalfont\normalsize\bfseries}{\thesubsection}{1em}{}
\titlespacing*{\subsection}{0pt}{3.25ex plus 1ex minus .2ex}{0.5em}

\newcommand{\periodafter}[1]{#1.}
\titleformat{\subsubsection}[runin]{\normalfont\normalsize\bfseries\it}{\thesubsubsection}{1em}{\periodafter}

\titlespacing*{\subsubsection}{0pt}{3.25ex plus 1ex minus .2ex}{0.5em}

There were also various other minor tweaks, like using Times New Roman, sorted with
\usepackage{times}, and changing the book class options to \documentclass[12pt,a4paper,oneside]{book}.

Viva success!

After much angst over the years, I successfully defended my PhD thesis today! I have a few corrections to do but I aim to finish them by Christmas. I am so relieved!

The reality hasn’t quite hit me yet, but I am thrilled.