Introduction

What is the Super Graphing Data Logger (SGDL)? It is an Arduino project that integrates data logging and the graphing of this data online using little more than an Arduino with the appropriate shields and sensors.   It differs from similar projects in that it doesn’t require a separate server or system to collect the data or to run script for the actual plot. Between the Arduino and the user’s browser, everything is taken care of.

Some time back I came across this neat javaScript based library for plotting and graphing called Highcharts JS. It didn’t take long for me to realize that charting with javaScript is very convenient for projects in which the server is limited in it’s capabilities, such as when using an Arduino with the Ethernet shield. Since the user’s browser does all the heavy lifting, the Arduino only needs to serve the files which is something it is perfectly capable of. This is especially true now that the Ethernet and SD libraries included in 1.0 support opening of multiple files simultaneously amongst other things. Thus the use of Highcharts allows us to create beautiful interactive charts based on data logged by the Arduino using nothing but the Arduino (and your browser, and a public javaScript CDN).

The Results

The best way to appreciate the final product is to actually play with it. While I’m not going to open up my home network and Arduino to the big wide internet, I have mirrored the pages and datafiles it produces on the webhost I used for my Has the World Ended Yet? project. You can find them here. These won’t be updated with new datapoints like the actual Arduino version will be, but they should at least give a fair impression of how the project looks and feels without the need to actually implement it.

For those who are unsure what they are looking at, I’ll offer a quick interpretation:

The list of data files available for graphing.

Going to the above page, we see that we are presented with a very basic list of the data files that can be selected from. Clicking any of them will cause  the graph for that datafile to be loaded (much more quickly than the Arduino can manage).

A graph for the first week of data collected.

This chart for the 25-12-12.CSV file is already complete, and won’t have any new data added to it in the future, because the files for subsequent weeks have already been made. There is a lot to see though. The two data points that are at 1000 on the y-axis are from when I pointed a bright flashlight directly at the photo sensor. All of the data points between 300 and 400 on the y-axis are the result of the basement lights being on. The abnormally large gaps in the data are periods when the Arduino was powered off because I was still tweaking and developing it. Finally, the short humps that occur everyday are the result of natural light coming through one of the basement windows. By zooming in on one of them, we can see even more detail:

The intensity of natural light in my basement.

The first thing we notice is that the levels rise from zero to about 65 before falling and levelling out at close to 35 for two hours. This is followed by a another small increase before it ultimately decreases down to a value of ~10 where it levels out. That middle valley where the light levels are equal to 35 is due to the shadow cast  on the basement window by our neighbour’s house to the south of us. The levelling out of the light intensity at 10 after all the daylight has disappeared is because a light out in the hallway is usually on in the evening. It is eventually turned off for the night, causing the light levels to drop to zero where they will usually remain until the next morning. I must admit, I’m impressed that the cheap $1.00 photoresistor is capable of capturing this level of detail, and that these trends are so easily interpreted from the graphs.

How to Make One For Yourself

To replicate this project, a few things are necessary. You’ll obviously need an Arduino capable of connecting over Ethernet and storing files on an SD card. In my case, this is achieved through the use of an Uno with the Ethernet shield. Presumably an Arduino Ethernet model will also work fine, though I have not personally tested it. Other non official Ethernet shields and SD card adapters may also work if they use the same libraries, though I make no guarantees. For the more adventurous, it may be possible to adapt my code to achieve the same functionality using a Wifi shield. You will also need a data source of some sort. For my project I chose to use a very cheap photoresistor, which I rigged up on a small perf board to plug directly into the 5v, gnd, and A0 pins of my Arduino (or more precisely,  the headers on the Ethernet shield). It is set up in such a way that the minimum recordable light intensity is zero, while the maximum is 1024.

The photo sensor board fits like a charm.

One header is bent to reach A0.

The pins on the male headers don’t quite line up, so I intentionally used extra long ones and added a slight S-curve to the one that goes to A0. This can be seen in better detail above. For those who are interested, the circuit is very simple:

The circuit.

Before we get started, we need to make sure our SD card is good to go. It should be formatted as a FAT16 or FAT32 filesystem, the details of which are available on the official Arduino website. Once that is done, we need to ensure two things are present in the root directory of the card: the HC.htm file, and a data/ directory for our datafiles. The data directory is easily made with the same computer that was used to format the card provided one has an SD card reader of some sort. The HC.htm simply consists of the following code:

<!DOCTYPE HTML>
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
        <title>Super Graphing Data Logger!</title>

        <script type="text/javascript" src="http://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script>
        <script type="text/javascript">
function getDataFilename(str){
    point = str.lastIndexOf("file=")+4;

    tempString = str.substring(point+1,str.length)
    if (tempString.indexOf("&") == -1){
    return(tempString);
    }
    else{
        return tempString.substring(0,tempString.indexOf("&"));
    }
        
}

query  = window.location.search;

var dataFilePath = "/data/"+getDataFilename(query);

$(function () {
    var chart;
    $(document).ready(function() {
    
        // define the options
        var options = {
    
            chart: {
                renderTo: 'container',
                zoomType: 'x',
                spacingRight: 20
            },
    
            title: {
                text: 'Light levels recorded by the Arduino'
            },
    
            subtitle: {
                text: 'Click and drag in the plot area to zoom in'
            },
    
            xAxis: {
                type: 'datetime',
                maxZoom: 2 * 3600000
            },
    
            yAxis: {
                title: {
                    text: 'Light Levels (0 - 1024)'
                },
                min: 0,
                startOnTick: false,
                showFirstLabel: false
            },
    
            legend: {
                enabled: false
            },
    
            tooltip: {
                formatter: function() {
                        return '<b>'+ this.series.name +'</b><br/>'+
                        Highcharts.dateFormat('%H:%M - %b %e, %Y', this.x) +': '+ this.y;
                }
            },
    
            plotOptions: {
                series: {
                    cursor: 'pointer',
                    lineWidth: 1.0,
                    point: {
                        events: {
                            click: function() {
                                hs.htmlExpand(null, {
                                    pageOrigin: {
                                        x: this.pageX,
                                        y: this.pageY
                                    },
                                    headingText: this.series.name,
                                    maincontentText: Highcharts.dateFormat('%H:%M - %b %e, %Y', this.x) +':<br/> '+
                                        this.y,
                                    width: 200
                                });
                            }
                        }
                    },
                }
            },
    
            series: [{
                name: 'Light Levels',
                marker: {
                    radius: 2
                }
            }]
        };
    
    
        // Load data asynchronously using jQuery. On success, add the data
        // to the options and initiate the chart.
        // http://api.jquery.com/jQuery.get/
        jQuery.get(dataFilePath, null, function(csv, state, xhr) {
            var lines = [],
                date,
    
                // set up the two data series
                lightLevels = [];
    
            // inconsistency
            if (typeof csv !== 'string') {
                csv = xhr.responseText;
            }
    
            // split the data return into lines and parse them
            csv = csv.split(/\n/g);
            jQuery.each(csv, function(i, line) {
    
                // all data lines start with a double quote
                line = line.split(',');
                date = parseInt(line[0], 10)*1000;
    
                lightLevels.push([
                    date,
                    parseInt(line[1], 10)
                ]);
                
            });
    
            options.series[0].data = lightLevels;
    
            chart = new Highcharts.Chart(options);
        });
    });
    
});
        </script>
    </head>
    <body>
        <p style="text-align:center;">Please allow the chart to load, it may take up to 30 seconds </p>
        <hr/>
<script src="http://cdnjs.cloudflare.com/ajax/libs/highcharts/2.3.5/highcharts.js"></script>

<!-- Additional files for the Highslide popup effect -->
<script type="text/javascript" src="http://www.highcharts.com/highslide/highslide-full.min.js"></script>
<script type="text/javascript" src="http://www.highcharts.com/highslide/highslide.config.js" charset="utf-8"></script>
<link rel="stylesheet" type="text/css" href="http://www.highcharts.com/highslide/highslide.css" />

<div id="container" style="min-width: 400px; height: 400px; margin: 0 auto"></div>

    </body>
</html>

You will need to edit this file first to make sure it points towards the preferred  location of your highcharts.js files. You can leave this as the public CDN: http://cdnjs.cloudflare.com/ajax/libs/highcharts/2.3.5/highcharts.js, change it to point towards your own webhost, or it can even be on the Arduino’s SD card (this will be slow). It is not necessary to create a datafile before hand, the SGDL sketch will take care of that when it decides to record its first data point. Before we get that far though, it is necessary to make sure we have configured the EEPROM memory for the SGDL sketch. This is very easily accomplished using a separate sketch, which I have called EEPROM_config. This sketch (along with SGDL itself) requires an extra library called EEPROMAnything, which needs to be added to the Arduino’s libraries folder wherever one’s sketchbook folder is. While you’re at it, you should also add the Time library which we need for SGDL.

/* ************************************************************************
 * ***            Super Graphing Data Logger - EEPROM config            ***
 * ************************************************************************
 * Everett Robinson, December 2012.
 *
 * The following extra non standard libraries were used, and will need to be
 * added to the libraries folder:
 * - EEPROMAnything: http://playground.arduino.cc/Code/EEPROMWriteAnything
 *
 * This sketch helps you set the values in EEPROM which are necessary for
 * Super Graphing Data Logger. It should only need the be run once before
 * the first time you set up SGDL, or in the unlikely event that the EEPROM
 * becomes corrupted.
 *
 * Please ensure that the values in configuration config are appropriate for
 * your project before uncommenting the EEPROM_writeAnything(0, config); line.
 *
 */

#include <EEPROM.h>
#include <EEPROMAnything.h>

typedef struct{
    unsigned long newFileTime;
    char workingFilename[19];
  } configuration;

//This is a one off thing, so everything is in setup
void setup(){
  Serial.begin(9600);
  
  //Create the config struct to write to EEPROM, change values as appropriate
  //Make sure your filename is not too long for the workingFilename char array 
  configuration config = {1356912000L,"/data/25-12-12.csv"};
  //Write the values to the EEPROM
  //EEPROM_writeAnything(0, config);       //Uncomment when you're sure everything is correct
  configuration config2;                   //Create a second config struct for verification
  EEPROM_readAnything(0,config2);
  Serial.print("The value read from EEPROM for newFileTime is: ");
  Serial.println(config2.newFileTime);
  Serial.print("The value read from EEPROM for workingFilename is: ");
  Serial.println(config2.workingFilename);
  Serial.println("If those values are correct then everything went as planned. Otherwise,");
  Serial.println("please double check that the values declared for the struct config are");
  Serial.println("correct and that that EEPROM_writeAnything line is uncommented.");
}


void loop(){
}

I have intentionally commented out the write line so that no one writes junk to the EEPROM by accident. While the EEPROM has a life of ~100,000 write cycles, I’d rather not waste any of them. Please review the sketch carefully and ensure you’ve adjusted it accordingly before uploading it to the Arduino. The most important thing is to ensure that your newFileTime is something sensible (in the near future most of all).

Now that that’s all taken care of, we’re ready to get SGDL all set up! The code will need a few adjustments for your own specific setup, mostly in regards to the Ethernet MAC and IP addresses. I trust that anyone making use of this code already knows how to configure their router to work with the Arduino, and that they can find the appropriate local IP address to update this sketch with. You may also wish to change the timeserver IP address to one that is geographically closer to yourself.

I currently have my code set up to make a measurement every 10 minutes, and to create a new data file every week. You are welcome to change those parameters, just be aware that the current data file management names files using a dd-mm-yy.csv date format, so the new file interval should be at least 24 hours. Another concern, is that the shorter the measurement interval and the longer the new data file interval is, the larger the files will be. Because the Arduino is not especially powerful, this will have consequences for the loading times of each chart.

/* ************************************************************************
 * ***                    Super Graphing Data Logger                    ***
 * ************************************************************************
 * Everett Robinson, December 2012. More at: http://everettsprojects.com
 *
 * This sketch relies on the SD and ethernet libraries in arduino 1.0 or newer.
 * The following extra non standard libraries were also used, and will need to
 * be added to the libraries folder:
 * - Time: http://playground.arduino.cc/Code/Time
 * - EEPROMAnything: http://playground.arduino.cc/Code/EEPROMWriteAnything
 *
 * If this is your first time setting up this project, please go get the
 * EEPROM_config sketch from http://everettsprojects.com so that you can 
 * configure the config struct in the EEPROM memory. Usage of the EEPROM 
 * is needed to make the project resiliant against a temporary loss of power.
 *
 * You must also ensure that you have the HC.htm file in the root directory
 * of your SD card, as well as a data directory where the datafiles will be
 * stored.
 *
 * This sketch combines the functionality of an existing fileserver example
 * which can be found at http://www.ladyada.net/learn/arduino/ethfiles.html
 * with the Datalogger example that comes with the new SD library from 1.0,
 * as well as some code from the UdpNtpClient example that cones with the
 * ethernet library. 
 *
 * Added to all of these are some tricks to make it manage and serve up the
 * datafiles in conjunction with a page which uses highcharts JS to graph it.
 * This is basically accomplished using the arduino by itself. Because I
 * actually host the highcharts.js files externally, this is true more in
 * theory than in actual practice, but oh well. It should work just fine to
 * have the highcharts.js file on the arduino's SD card, though loading the 
 * page will be painfully slow.
 *
 * Some of the code this was derived from may or may not be under a GPL
 * licence; I'm not entirely sure. I suppose anyone using this should treat 
 * it like it is too, but I don't really care too much.
 * Also if one intends to use this for commercial applications, it may be
 * necessary to purchase a license for Highcharts.
 *
 * Changes:   -------------------------------------------------------------
 * January 2013: Updated so that the dd-mm-yy.csv file format is properly 
 * followed, all single digit days, months, and years will have a leading 
 * zero now. 
 *
 */

#include <SD.h>
#include <Ethernet.h>
#include <EthernetUdp.h>
#include <SPI.h>
#include <string.h>
#include <Time.h>
#include <EEPROM.h>
#include <EEPROMAnything.h>
#include <avr/pgmspace.h>

/************ ETHERNET STUFF ************/
byte mac[] = { 0x90, 0xA2, 0xDA, 0x00, 0x4C, 0x64 };
byte ip[] = { 192,168,1, 100 };
EthernetServer server(80);

/************** NTP STUFF ***************/
unsigned int localPort = 8888;          // local port to listen for UDP packets
IPAddress timeServer(132, 163, 4, 101); //NIST time server IP address: for more info
                                        //see http://tf.nist.gov/tf-cgi/servers.cgi

const int NTP_PACKET_SIZE= 48; //NTP time stamp is in the first 48 bytes of the message
byte packetBuffer[ NTP_PACKET_SIZE]; //buffer to hold incoming and outgoing packets 
EthernetUDP Udp;

/*** DATA LOGGER AND TIMER CONTROLS ****/
const int analogPin = 0;
unsigned long lastIntervalTime = 0; //The time the last measurement occured.
#define MEASURE_INTERVAL 600000     //10 minute intervals between measurements (in ms)
unsigned long newFileTime;          //The time at which we should create a new week's file
#define FILE_INTERVAL 604800        //One week worth of seconds

//A structure that stores file config variables from EEPROM
typedef struct{                     
    unsigned long newFileTime;      //Keeps track of when a newfile should be made.
    char workingFilename[19];       //The path and filename of the current week's file
} configuration;
  
configuration config;               //Actually make our config struct


// Strings stored in flash mem for the Html Header (saves ram)
prog_char HeaderOK_0[] PROGMEM = "HTTP/1.1 200 OK";            //
prog_char HeaderOK_1[] PROGMEM = "Content-Type: text/html";    //
prog_char HeaderOK_2[] PROGMEM = "";                           //

// A table of pointers to the flash memory strings for the header
PROGMEM const char *HeaderOK_table[] = {   
  HeaderOK_0,
  HeaderOK_1,
  HeaderOK_2
};

// A function for reasy printing of the headers  
void HtmlHeaderOK(EthernetClient client) {
  
    char buffer[30]; //A character array to hold the strings from the flash mem
    
    for (int i = 0; i < 3; i++) {
      strcpy_P(buffer, (char*)pgm_read_word(&(HeaderOK_table[i]))); 
      client.println( buffer );
    }
} 
  
  
// Strings stored in flash mem for the Html 404 Header
prog_char Header404_0[] PROGMEM = "HTTP/1.1 404 Not Found";     //
prog_char Header404_1[] PROGMEM = "Content-Type: text/html";    //
prog_char Header404_2[] PROGMEM = "";                           //
prog_char Header404_3[] PROGMEM = "<h2>File Not Found!</h2>"; 

// A table of pointers to the flash memory strings for the header
PROGMEM const char *Header404_table[] = {   
  Header404_0,
  Header404_1,
  Header404_2,
  Header404_3
};

// Easy peasy 404 header function
void HtmlHeader404(EthernetClient client) {
  
    char buffer[30]; //A character array to hold the strings from the flash mem
    
    for (int i = 0; i < 4; i++) {
      strcpy_P(buffer, (char*)pgm_read_word(&(Header404_table[i]))); 
      client.println( buffer );
    }
} 


void setup() {
  Serial.begin(9600);
  
  pinMode(10, OUTPUT);          // set the SS pin as an output (necessary!)
  digitalWrite(10, HIGH);       // but turn off the W5100 chip! 
  
  // see if the card is present and can be initialized:
  if (!SD.begin(4)) {
    Serial.println("Card failed, or not present");
    // don't do anything more:
    return;
  }
  Serial.println("card initialized.");
  
  // The SD card is working, start the server and ethernet related stuff!
  Ethernet.begin(mac, ip);
  server.begin();
  Udp.begin(localPort);
  EEPROM_readAnything(0,config); // make sure our config struct is syncd with EEPROM
}


// A function that takes care of the listing of files for the
// main page one sees when they first connect to the arduino.
// it only lists the files in the /data/ folder. Make sure this
// exists on your SD card.
void ListFiles(EthernetClient client) {
  
  File workingDir = SD.open("/data");
  
  client.println("<ul>");
  
    while(true) {
      File entry =  workingDir.openNextFile();
       if (! entry) {
         break;
       }
       client.print("<li><a href=\"/HC.htm?file=");
       client.print(entry.name());
       client.print("\">");
       client.print(entry.name());
       client.println("</a></li>");
       entry.close();
    }
  client.println("</ul>");
  workingDir.close();
}

// A function to get the Ntp Time. This is used to make sure that the data
// points recorded by the arduino are referenced to some meaningful time
// which in our case is UTC represented as unix time (choosen because it 
// works simply with highcharts without too much unecessary computation).
unsigned long getTime(){
  sendNTPpacket(timeServer); // send an NTP packet to a time server

  // wait to see if a reply is available
  delay(1000);  
  if ( Udp.parsePacket() ) {  
    // We've received a packet, read the data from it
    Udp.read(packetBuffer,NTP_PACKET_SIZE);  // read the packet into the buffer

    //the timestamp starts at byte 40 of the received packet and is four bytes,
    // or two words, long. First, esxtract the two words:

    unsigned long highWord = word(packetBuffer[40], packetBuffer[41]);
    unsigned long lowWord = word(packetBuffer[42], packetBuffer[43]);  
    // combine the four bytes (two words) into a long integer
    // this is NTP time (seconds since Jan 1 1900):
    unsigned long secsSince1900 = highWord << 16 | lowWord;  
    // Unix time starts on Jan 1 1970. In seconds, that's 2208988800:
    const unsigned long seventyYears = 2208988800UL;     
    // subtract seventy years:
    unsigned long epoch = secsSince1900 - seventyYears;  
    // return Unix time:
    return epoch;
  }
}

// send an NTP request to the time server at the given address,
// necessary for getTime().
unsigned long sendNTPpacket(IPAddress& address){
  
  // set all bytes in the buffer to 0
  memset(packetBuffer, 0, NTP_PACKET_SIZE); 
  // Initialize values needed to form NTP request
  // (see URL above for details on the packets)
  packetBuffer[0] = 0b11100011;   // LI, Version, Mode
  packetBuffer[1] = 0;     // Stratum, or type of clock
  packetBuffer[2] = 6;     // Polling Interval
  packetBuffer[3] = 0xEC;  // Peer Clock Precision
  // 8 bytes of zero for Root Delay & Root Dispersion
  packetBuffer[12]  = 49; 
  packetBuffer[13]  = 0x4E;
  packetBuffer[14]  = 49;
  packetBuffer[15]  = 52;

  // all NTP fields have been given values, now
  // you can send a packet requesting a timestamp:         
  Udp.beginPacket(address, 123); //NTP requests are to port 123
  Udp.write(packetBuffer,NTP_PACKET_SIZE);
  Udp.endPacket(); 
}


// How big our line buffer should be for sending the files over the ethernet.
// 75 has worked fine for me so far.
#define BUFSIZ 75

void loop(){
  if ((millis() % lastIntervalTime) >= MEASURE_INTERVAL){ //Is it time for a new measurement?
     
    char dataString[20] = "";
    int count = 0;
    unsigned long rawTime;
    rawTime = getTime();

    while((rawTime == 39) && (count < 12)){     //server seems to send 39 as an error code
      delay(5000);                              //we want to retry if this happens. I chose
      rawTime = getTime();                      //12 retries because I'm stubborn/persistent.
      count += 1;                               //NIST considers retry interval of <4s as DoS
    }                                           //attack, so fair warning.
    
    if (rawTime != 39){                         //If that worked, and we have a real time
      
      //Decide if it's time to make a new file or not. Files are broken
      //up like this to keep loading times for each chart bearable.
      //Lots of string stuff happens to make a new filename if necessary.
      if (rawTime >= config.newFileTime){
        int dayInt = day(rawTime);
        int monthInt = month(rawTime);
        int yearInt = year(rawTime);
        char newFilename[18] = "";
        char dayStr[3];
        char monthStr[3];
        char yearStr[5];
        char subYear[3];
        strcat(newFilename,"data/");
        itoa(dayInt,dayStr,10);
        if (dayInt < 10){
          strcat(newFilename,"0");
        }
        strcat(newFilename,dayStr);
        strcat(newFilename,"-");
        itoa(monthInt,monthStr,10);
        if (monthInt < 10){
          strcat(newFilename,"0");
        }
        strcat(newFilename,monthStr);
        strcat(newFilename,"-");
        itoa(yearInt,yearStr,10);
        //we only want the last two digits of the year
        memcpy( subYear, &yearStr[2], 3 );
        strcat(newFilename,subYear);
        strcat(newFilename,".csv");
        
        //make sure we update our config variables:
        config.newFileTime += FILE_INTERVAL;
        strcpy(config.workingFilename,newFilename);
        //Write the changes to EEPROM. Bad things may happen if power is lost midway through,
        //but it's a small risk we take. Manual fix with EEPROM_config sketch can correct it.
        EEPROM_writeAnything(0, config); 
      }
        
      //get the values and setup the string we want to write to the file
      int sensor = analogRead(analogPin);  
      char timeStr[12];
      char sensorStr[6];
      
      ultoa(rawTime,timeStr,10); 
      itoa(sensor,sensorStr,10);
      
      strcat(dataString,timeStr);
      strcat(dataString,",");
      strcat(dataString,sensorStr);
      
      //open the file we'll be writing to.
      File dataFile = SD.open(config.workingFilename, FILE_WRITE);
  
      // if the file is available, write to it:
      if (dataFile) {
        dataFile.println(dataString);
        dataFile.close();
        // print to the serial port too:
        Serial.println(dataString);
      }  
      // if the file isn't open, pop up an error:
      else {
        Serial.println("Error opening datafile for writing");
      }
    }
    else{
      Serial.println("Couldn't resolve a time from the Ntp Server.");
    }
    //Update the time of the last measurment to the current timer value
    lastIntervalTime = millis();
  }
  //No measurements to be made, make sure the webserver is available for connections.
  else{
    char clientline[BUFSIZ];
    int index = 0;
    
    EthernetClient client = server.available();
    if (client) {
      // an http request ends with a blank line
      boolean current_line_is_blank = true;
      
      // reset the input buffer
      index = 0;
      
      while (client.connected()) {
        if (client.available()) {
          char c = client.read();
          
          // If it isn't a new line, add the character to the buffer
          if (c != '\n' && c != '\r') {
            clientline[index] = c;
            index++;
            // are we too big for the buffer? start tossing out data
            if (index >= BUFSIZ) 
              index = BUFSIZ -1;
            
            // continue to read more data!
            continue;
          }
          
          // got a \n or \r new line, which means the string is done
          clientline[index] = 0;
          
          // Print it out for debugging
          Serial.println(clientline);
          
          // Look for substring such as a request to get the root file
          if (strstr(clientline, "GET / ") != 0) {
            // send a standard http response header
            HtmlHeaderOK(client);
            // print all the data files, use a helper to keep it clean
            client.println("<h2>View data for the week of (dd-mm-yy):</h2>");
            ListFiles(client);
          }
          else if (strstr(clientline, "GET /") != 0) {
            // this time no space after the /, so a sub-file!
            char *filename;
            
            filename = strtok(clientline + 5, "?"); // look after the "GET /" (5 chars) but before
            // the "?" if a data file has been specified. A little trick, look for the " HTTP/1.1"
            // string and turn the first character of the substring into a 0 to clear it out.
            (strstr(clientline, " HTTP"))[0] = 0;
            
            // print the file we want
            Serial.println(filename);
            File file = SD.open(filename,FILE_READ);
            if (!file) {
              HtmlHeader404(client);
              break;
            }
            
            Serial.println("Opened!");
                      
            HtmlHeaderOK(client);
            
            int16_t c;
            while ((c = file.read()) > 0) {
                // uncomment the serial to debug (slow!)
                //Serial.print((char)c);
                client.print((char)c);
            }
            file.close();
          }
          else {
            // everything else is a 404
            HtmlHeader404(client);
          }
          break;
        }
      }
      // give the web browser time to receive the data
      delay(1);
      client.stop();
    }
  }
}

*****

Installing Ubuntu and some basic configuration

To start it’s necessary to get your preferred Linux distribution running on the BeagleBone. I chose Ubuntu mainly because I found an easy and ready to install image, though the fact that it is compiled to take advantage of the hard float capabilities in this arm processor was also a factor. The following instructions were up to date when I utilized them, though it’s entirely possible that a newer image has been created in the meantime and you should use that one if possible. You will also need a microSD card to write the image to (I used a 4GB card, though a 2GB card or greater should work).

Credit goes to http://elinux.org/BeagleBoardUbuntu#Demo_Image as the basis for this portion of my guide.

The first thing we need to do is download the operating system image that we will write to the microSD card:

wget http://rcn-ee.net/deb/rootfs/precise/ubuntu-12.04-r1-minimal-armhf.tar.xz

This image needs to be unpacked before we write it to the SD card:

tar xJf ubuntu-12.04-r1-minimal-armhf.tar.xz
cd ubuntu-12.04-r1-minimal-armhf

We need to know where to install this image , and even though it is likely /dev/mmcblk0 we should check anyway:

sudo ./setup_sdcard.sh –probe-mmc

In the output a line like Disk /dev/mmcblk0: 3957 MB, 3957325824 bytes should be seen. If there is only one line starting with Disk /dev/mmcblk… then that is the SD card we want, and that’s the location you should use. If there are more than one of these /dev/mmcblk lines, then you must figure out which one belongs to the SD card you want to write to, otherwise you may overwrite something important.

Having figured out where the SD card is mounted, we will run the script to write the contents of our image to it:

sudo ./setup_sdcard.sh –mmc /dev/mmcblk0 –uboot bone

Where /dev/mmcblk0 may be something else if you found the SD card to be mounted somewhere else in the previous step.
At this point you only have to insert the microSD card and plug it into the BeagleBone to get started. Because the board lacks a way to hook up a display it is easiest to insert an ethernet cable and remotely SSH into it with the user “ubuntu” and the password “temppwd”

ssh ubuntu@<Local IP Address>

I have managed to use the DHCP reservation functionality on my home router to ensure that the BeagleBone has a fixed local IP address, and I strongly encourage you to do the same if possible. Because this process varies by router manufacturer and model, I cannot offer much guidance in this regard. This guide may be of some help.

Our first step after logging in should be to change the password for this account:

passwd

This will ask for the current password, and the one you wish replace it with twice. The next two steps will create a new user account with a better name (just replace <username> with your preference) and give it administrative privileges. These steps are cosmetic, and are unnecessary if you just wish to use the account “ubuntu”.

sudo adduser <username>

sudo usermod -aG admin <username>

The next two commands will allow us to change the host name for the BeagleBone to something other than “omap” and are also optional as they are purely cosmetic.

sudo nano /etc/hostname

You should change the first line of this file to whatever you’d like as a new host name (don’t use spaces) and then press CTRL-o to save the changes and CTRL-x to exit. We will also need to change the following file:

sudo nano /etc/hosts

The word omap in the second line of this file should be replaced with the same host name we just used.

I encountered perl locale errors when I tried doing various things later on, which may still be an issue. To fix them you want to check that /etc/default/locale contains a few things:

sudo nano /etc/default/locale

You should find the following lines within, though they do not necessarily need to be US english:

LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8:en
LC_ALL=en_US.UTF-8

If any of these lines are not present, then you should add them (substituting in another language pack such as en_GB if you’re so inclined.)

Before proceeding, it is also a good idea to make sure all of the packages on your system are up to date:

sudo apt-get update
sudo apt-get upgrade

sudo reboot

*****

Locking down SSH

Credit goes to http://www.debian-administration.org/articles/530 as the basis for this portion of my guide.
We need to generate a set of public and private keys on the computer you wish to SSH into the BeagleBone from. It is important to make sure you are running this locally on the client computer, not through SSH on the BeagleBone:

ssh-keygen

You will then be prompted for a password to protect the key we just generated. This password is not transmitted in any way to the system were are using SSH to access, but rather to decrypt the key on your local machine. If you leave the password blank it will not encrypt this key and make it so that anyone with the right file permissions can see it.
Now we want to install the key on the BeagleBone:

ssh-copy-id -i .ssh/id_rsa.pub <username>@<local IP address>

If this was successful, we should be able to SSH into the BeagleBone without entering a password.
We should then check that we only installed the key we intended to:

nano ~/.ssh/authorized_keys

You should see only one line in the file.

Next, it is a good idea to disable password authentication and limit the users who can access the board via SSH.
You do not want to do this by disabling password authentication for the chosen account as outlined in the debian-administration.org article; this runs the risk of breaking your ability to use sudo and can leave you without root access. I may or may not have made this error first hand. It’s not fun. We achieve the desired effect by editing /etc/ssh/sshd_config .

sudo nano /etc/ssh/sshd_config

First things first, it’s never a bad idea to disable root login. This isn’t really necessary on an Ubuntu system since there is no functioning root password and everything is done using sudo, but we’re better safe than sorry. Find the line beginning with PermitRootLogin and change it to no:

PermitRootLogin no

We also want to disable password authentication by finding the line #PasswordAuthentication yes, changing it to no, and uncommenting it by removing the #:

PasswordAuthentication no

Because we do not need X11-forwarding for our purposes you may also wish to change that line to:

X11Forwarding no

And to limit the users allowed to SSH into the BeagleBone we want to add the following line to the end of the file, using the username we created earlier in the place of <username>:

AllowUsers <username>

Finally, to finish our lockdown of SSH, we need to restart it so that the changes can take effect:

sudo service ssh restart

*****

Getting rtorrent and rutorrent running

This portion of my guide has been mainly adapted from http://forums.rutorrent.org/index.php?topic=256.0.

First we want to install a large number of packages:

sudo apt-get install apache2 apache2.2-common apache2-utils autoconf automake autotools-dev binutils build-essential bzip2 ca-certificates comerr-dev cpp cpp-4.6 dpkg-dev file g++ g++-4.6 gawk gcc gcc-4.6 libapache2-mod-php5 libapache2-mod-scgi libapr1 libaprutil1 libc6-dev libcppunit-dev libcurl3 libcurl4-openssl-dev libexpat1 libidn11 libidn11-dev libkdb5-6 libgssrpc4 libkrb5-dev libmagic1 libncurses5 libncurses5-dev libneon27 libpcre3 libpq5 libsigc++-2.0-dev libsqlite0 libsqlite3-0 libssl-dev libstdc++6-4.6-dev libsvn1 libtool libxml2 linux-libc-dev lynx m4 make mime-support ntp ntpdate openssl patch perl perl-modules php5 php5-cgi php5-cli php5-common php5-curl php5-dev php5-geoip php5-sqlite php5-xmlrpc pkg-config python-scgi screen sqlite ssl-cert subversion ucf unrar zlib1g-dev pkg-config unzip htop screen libwww-perl curl

Several of these packages are probably already installed, though it does not hurt to make sure. Some of the packages in this long list have been updated to newer versions since the guide I have adapted thus from was written. If any appreciable length of time has passed since I wrote this, then you are well advised to check if any of these packages need to have their version numbers changed. If this is the case, then apt-get is likely to spit out some error messages about certain packages being missing. The specific packages I updated were: cpp-4.6, g++-4.6, gcc-4.6, libstdc++6-4.6-dev, libkdb5-6, and libneon27. Here’s a quick way to check the version numbers for packages in the ubuntu repository online.

Having everything we need, our next step is to configure apache:

a2enmod ssl
a2enmod auth_digest
a2enmod scgi

We need to make sure that apache has SCGI support enabled so that the rutorrent webui will work:

sudo nano /etc/apache2/apache2.conf

You then need to paste the following at the end of the file:

SCGIMount /RPC2 127.0.0.1:5000
servername localhost

To make sure everything we’ve done takes effect it is a good idea to restart the server:

sudo reboot

To make sure everything is working, on your client machine type the local ip address assigned to your BeagleBone into the address bar of your browser:

http:// < Local IP Address>

You should see the following:

Because our rutorrent front end to rtorrent will be password protected, we need to have HTTPS functionality eneabled. To achieve this we need an SSL certificate. This process will ask for a lot of information which you can fill in however you see fit, though it’s a good idea to use the domain name your BeagleBone will be connected to if you have one:

openssl req $@ -new -x509 -days 365 -nodes -out /etc/apache2/apache.pem -keyout /etc/apache2/apache.pem
chmod 600 /etc/apache2/apache.pem

This is a self signed certificate, meaning your browser will probably spit out a warning the first time you connect. Just ignore the warning and store the exception, and this won’t happen again. Our next step is to protect our apache webserver with a username and password:

sudo htdigest -c /etc/apache2/passwords gods <webusername>

This username, <webusername>, and password can be whatever you like. It can even be the same user and password we used earlier when setting sup the system. I personally decided to use a different username and password because it offers a slight security advantage in that if someone figures out the password to our rutorrent and apache setup, they still don’t have the password for root privileges through the operating system account we made earlier. It is unlikely that this scenario would ever happen since we made access through ssh only possible using key authentication, though it’s not much of a hassle for a little added security (Besides, I know you’ll just save the webusername and password in your browser anyway).

We also need to configure apache using the /etc/apache2/sites-available/default file:

sudo nano /etc/apache2/sites-available/default

You want to replace the contents of this file with the following, where the two instances of <Local IP Address> are replaced by the address of your BeagleBone on the local network:

<VirtualHost *:80>
	ServerAdmin webmaster@localhost

	DocumentRoot /var/www/
	<Directory />
		Options FollowSymLinks
		AllowOverride None
	</Directory>
	<Directory /var/www/>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride None
		Order allow,deny
		allow from all
	</Directory>

	ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
	<Directory "/usr/lib/cgi-bin">
		AllowOverride None
		Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
		Order allow,deny
		Allow from all
	</Directory>

	ErrorLog /var/log/apache2/error.log

	# Possible values include: debug, info, notice, warn, error, crit,
	# alert, emerg.
	LogLevel warn

	CustomLog /var/log/apache2/access.log combined

	Alias /doc/ "/usr/share/doc/"
	<Directory "/usr/share/doc/">
		Options Indexes MultiViews FollowSymLinks
		AllowOverride None
		Order deny,allow
		Deny from all
		Allow from 127.0.0.0/255.0.0.0 ::1/128
	</Directory>

	<Location /rutorrent>
		AuthType Digest
		AuthName "gods"
		AuthDigestDomain /var/www/rutorrent/ http://<Local IP Address>/rutorrent

		AuthDigestProvider file
		AuthUserFile /etc/apache2/passwords
		Require valid-user
		SetEnv R_ENV "/var/www/rutorrent"
	</Location>

</VirtualHost>

<VirtualHost *:443>
	ServerAdmin webmaster@localhost

	SSLEngine on
	SSLCertificateFile /etc/apache2/apache.pem

	DocumentRoot /var/www/
	<Directory />
		Options FollowSymLinks
		AllowOverride None
	</Directory>
	<Directory /var/www/>
		Options Indexes FollowSymLinks MultiViews
		AllowOverride None
		Order allow,deny
		allow from all
	</Directory>

	ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
	<Directory "/usr/lib/cgi-bin">
		AllowOverride None
		Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
		Order allow,deny
		Allow from all
	</Directory>

	ErrorLog /var/log/apache2/error.log

	# Possible values include: debug, info, notice, warn, error, crit,
	# alert, emerg.
	LogLevel warn

	CustomLog /var/log/apache2/access.log combined

	Alias /doc/ "/usr/share/doc/"
	<Directory "/usr/share/doc/">
		Options Indexes MultiViews FollowSymLinks
		AllowOverride None
		Order deny,allow
		Deny from all
		Allow from 127.0.0.0/255.0.0.0 ::1/128
	</Directory>
	<Location /rutorrent>
		AuthType Digest
		AuthName "gods"
		AuthDigestDomain /var/www/rutorrent/ http://<Local IP Address>/rutorrent

		AuthDigestProvider file
		AuthUserFile /etc/apache2/passwords
		Require valid-user
		SetEnv R_ENV "/var/www/rutorrent"
	</Location>
</VirtualHost>

We then want to run the following to get apache running https:

sudo a2ensite default-ssl
sudo /etc/init.d/apache2 reload

If everything worked as intended, then going to https://&lt; Local IP Address> should show us the same page we saw earlier.

Unlike the guide I adapted these instructions from, I have decided not to install the Webmin configuration utilities for a couple of reasons. The first is that I wanted this project to help develop my skills as a Linux administrator, and a graphical GUI to change everything does not really fit that goal. The second reason is that the BeagleBone is not a powerful computer, and so I would prefer not to weigh it down with things that are not absolutely necessary.

Now that we have apache up and running with all the necessary bells and whistles, we can proceed to install and configure rtorrent and the rutorrent webui. At the time the guide I used as my starting point was written, the version of rtorrent in the Ubuntu repositories wasn’t complied with xmlrpc support, which was needed for rutorrent to work. This has long been fixed, and so you can probably just install the packages libxmlrpc-core-c3-dev and rtorrent. I can’t offer any guidance in this regard because I decided to compile the latest versions of these packages from source, mostly because I could. The compilation process will take a fair amount of time; likely more than an hour.

First things first, we need the sources for each of these packages:

cd ~/
mkdir source
cd source
svn co https://xmlrpc-c.svn.sourceforge.net/svnroot/xmlrpc-c/advanced/ xmlrpc-c
wget http://libtorrent.rakshasa.no/downloads/libtorrent-0.13.2.tar.gz
wget http://libtorrent.rakshasa.no/downloads/rtorrent-0.9.2.tar.gz
tar -xvzf libtorrent-0.13.2.tar.gz
tar -xvzf rtorrent-0.9.2.tar.gz
rm *.tar.gz

This simply downloads the sources, unpacks the archives and deletes the archive files once we have. These packages may be updated to newer versions by the time you read this, and you can change version numbers accordingly.

The first package we need to build is xmlrpc-c:

cd xmlrpc-c
./configure –disable-cplusplus
make
sudo make install

Once this has completed we will do the same for libtorrent, the backend of rtorrent:

cd ../libtorrent-0.13.2
./autogen.sh
./configure
make
sudo make install

And finally, we will compile rtorrent itself:

cd ../rtorrent-0.9.2
./autogen.sh
./configure –with-xmlrpc-c
make
sudo make install

sudo ldconfig

Next we need an rtorrent configuration file, which we will save as ~/.rtorrent.rc:

nano ~/.rtorrent.rc

and replace any contents with the following:

# This is an example resource file for rTorrent. Copy to
# ~/.rtorrent.rc and enable/modify the options as needed. Remember to
# uncomment the options you wish to enable.
#
# Based on original .rtorrent.rc file from http://libtorrent.rakshasa.no/
# Modified by Lemonberry for rtGui http://rtgui.googlecode.com/
#
# This assumes the following directory structure:
#
# /Torrents/Downloading - temporaray location for torrents while downloading (see "directory")
# /Torrents/Complete - Torrents are moved here when complete (see "on_finished")
# /Torrents/TorrentFiles/Auto - The 'autoload' directory for rtorrent to use.  Place a file
#           in here, and rtorrent loads it #automatically.  (see "schedule = watch_directory")
# /Torrents/Downloading/rtorrent.session - for storing rtorrent session information
#

# Maximum and minimum number of peers to connect to per torrent.
#min_peers = 40
max_peers = 100

# Same as above but for seeding completed torrents (-1 = same as downloading)
min_peers_seed = -1
max_peers_seed = -1

# Maximum number of simultanious uploads per torrent.
max_uploads = 10

# Global upload and download rate in KiB. "0" for unlimited.
download_rate = 0
upload_rate = 0

# Default directory to save the downloaded torrents.
directory = <directory torrents will download to by default>

# Default session directory. Make sure you don't run multiple instance
# of rtorrent using the same session directory. Perhaps using a
# relative path? (We disregard this because of our startup script later)
session = /home/<username>/.session

# Watch a directory for new torrents, and stop those that have been
# deleted.
#schedule = watch_directory,5,5,load_start=/home/downloads/<username>/watch/*.torrent
#schedule = untied_directory,5,5,stop_untied=

# Close torrents when diskspace is low. */
schedule = low_diskspace,5,60,close_low_diskspace=100M

# Stop torrents when reaching upload ratio in percent,
# when also reaching total upload in bytes, or when
# reaching final upload ratio in percent.
# example: stop at ratio 2.0 with at least 200 MB uploaded, or else ratio 20.0
#schedule = ratio,60,60,stop_on_ratio=200,200M,2000

# When the torrent finishes, it executes "mv -n  ~/Download/"
# and then sets the destination directory to "~/Download/". (0.7.7+)
# on_finished = move_complete,"execute=mv,-u,$d.get_base_path=,/home/downloads/<username>/complete/ ;d.set_directory=/home/downloads/<username>/complete/"

# The ip address reported to the tracker.
#ip = 127.0.0.1
#ip = rakshasa.no

# The ip address the listening socket and outgoing connections is
# bound to.
#bind = 127.0.0.1
#bind = rakshasa.no

# Port range to use for listening.
port_range = 55995-56000

# Start opening ports at a random position within the port range.
#port_random = yes

scgi_port = 127.0.0.1:5000

# Check hash for finished torrents. Might be usefull until the bug is
# fixed that causes lack of diskspace not to be properly reported.
#check_hash = no

# Set whetever the client should try to connect to UDP trackers.
#use_udp_trackers = no

# Alternative calls to bind and ip that should handle dynamic ip's.
#schedule = ip_tick,0,1800,ip=rakshasa
#schedule = bind_tick,0,1800,bind=rakshasa

# Encryption options, set to none (default) or any combination of the following:
# allow_incoming, try_outgoing, require, require_RC4, enable_retry, prefer_plaintext
#
# The example value allows incoming encrypted connections, starts unencrypted
# outgoing connections but retries with encryption if they fail, preferring
# plaintext to RC4 encryption after the encrypted handshake
#
encryption = allow_incoming,try_outgoing

# Enable DHT support for trackerless torrents or when all trackers are down.
# May be set to "disable" (completely disable DHT), "off" (do not start DHT),
# "auto" (start and stop DHT as needed), or "on" (start DHT immediately).
# The default is "off". For DHT to work, a session directory must be defined.
#
dht = disable

# UDP port to use for DHT.
#
# dht_port = 6881

# Enable peer exchange (for torrents not marked private)
#
peer_exchange = no

#
# Do not modify the following parameters unless you know what you're doing.
#

# Hash read-ahead controls how many MB to request the kernel to read
# ahead. If the value is too low the disk may not be fully utilized,
# while if too high the kernel might not be able to keep the read
# pages in memory thus end up trashing.
#hash_read_ahead = 10

# Interval between attempts to check the hash, in milliseconds.
#hash_interval = 100

# Number of attempts to check the hash while using the mincore status,
# before forcing. Overworked systems might need lower values to get a
# decent hash checking rate.
#hash_max_tries = 10

# Max number of files to keep open simultaniously.
#max_open_files = 128

# Number of sockets to simultaneously keep open.
#max_open_sockets =

# Example of scheduling commands: Switch between two ip's every 5
# seconds.
#schedule = "ip_tick1,5,10,ip=torretta"
#schedule = "ip_tick2,10,10,ip=lampedusa"

# Remove a scheduled event.
#schedule_remove = "ip_tick1"

There are a couple of lines you must replace with your own specific information. The first is directory = <directory torrents will download to by default> and the second is session = /home/<username>/.session . I also encourage you to do a little research so that you can change other settings to suit your purposes.

We need to make sure the directories we just told rtorrent to use exist. Because it would be silly to have your media stored on the same SD card containing the OS, I’ve attached an external USB hard drive to the BeagleBone and set my directory = <directory torrents will download to by default> line to point towards there. Since this is external media, we’ll need it to auto mount whenever the board reboots so that rtorrent will always be able to find it. We’ll start by making the directories for rtorrent and for the drive to be mounted to:

cd ~
sudo mkdir .session
sudo mkdir /media/<mount directory name>

I am mounting it in /media because that is the standard place to place such things in ubuntu, though you could just as well mount it in /mnt or another directory you have made. To achieve automagical mounting we will edit /etc/fstab after backing up the original:

sudo cp /etc/fstab /etc/fstab.backup

We’re going to mount the usb hard drive using the uuid rather than the /dev/sda path because it is a better way of identifying the drive we actually want. To find this uuid we need to run:

ls -l /dev/disk/by-uuid

One of the lines should have something like ../../sda1 listed in yellow, and a corresponding alphanumeric code in blue. The blue code is the uuid, which you should copy. We can now go ahead and edit /etc/fstab :

sudo nano /etc/fstab

We want to add a specific line to the end:

UUID=<uuid>     /media/<mount directory name> auto defaults 0 0

where <uuid> is the value we copied earlier, and /media/<mount directory name> is where we decided to mount the drive. If you happen to know the file system in use on your usb drive you may wish to change the auto in the above line to the proper filesystem. For ext4 this is just “ext4” and for ntfs (oh god, why?) it would be “ntfs-3g“, each without quotes. If using ntfs it may be necessary to install the ntfs-3g package.

Now that fstab has been modified, we want Ubuntu to recognize the changes:

sudo mount -a

And we will also make ourselves the owner of the external drive mount point:

sudo chown -R <username>:<username> /media/<mount directory name>

If everything went well, then we should find that rtorrent starts up without any issues:

rtorrent

Any problems should be noted in rtorrent with an error that should help direct you towards a solution. I got a warning about xmlrpc, though a little research showed this was just an advisory and nothing to actually be worried about. You can now quit rtorrent by pressing CTRL-q.

We also want rtorrent to start up automatically in such a way that it will also keep running when we are not logged in via ssh. This is accomplished using a startup script (which I have left unmodified for the original guide) that makes use of screen.

sudo nano /etc/init.d/rtorrent

Paste the follwing into that file. The only change necessary is to replace <username> in the line user=”<username>” with the user we created earlier:

#!/bin/sh
#############
###<Notes>###
#############
# This script depends on screen.
# For the stop function to work, you must set an
# explicit session directory using ABSOLUTE paths (no, ~ is not absolute) in your rtorrent.rc.
# If you typically just start rtorrent with just "rtorrent" on the
# command line, all you need to change is the "user" option.
# Attach to the screen session as your user with
# "screen -dr rtorrent". Change "rtorrent" with srnname option.
# Licensed under the GPLv2 by lostnihilist: lostnihilist _at_ gmail _dot_ com
##############
###</Notes>###
##############

#######################
##Start Configuration##
#######################
# You can specify your configuration in a different file
# (so that it is saved with upgrades, saved in your home directory,
# or whateve reason you want to)
# by commenting out/deleting the configuration lines and placing them
# in a text file (say /home/user/.rtorrent.init.conf) exactly as you would
# have written them here (you can leave the comments if you desire
# and then uncommenting the following line correcting the path/filename
# for the one you used. note the space after the ".".
# . /etc/rtorrent.init.conf

#Do not put a space on either side of the equal signs e.g.
# user = user
# will not work
# system user to run as
user="<username>"

# the system group to run as, not implemented, see d_start for beginning implementation
# group=`id -ng "$user"`

# the full path to the filename where you store your rtorrent configuration
config="`su -c 'echo $HOME' $user`/.rtorrent.rc"

# set of options to run with
options=""

# default directory for screen, needs to be an absolute path
base="`su -c 'echo $HOME' $user`"

# name of screen session
srnname="rtorrent"

# file to log to (makes for easier debugging if something goes wrong)
logfile="/var/log/rtorrentInit.log"
#######################
###END CONFIGURATION###
#######################
PATH=/usr/bin:/usr/local/bin:/usr/local/sbin:/sbin:/bin:/usr/sbin
DESC="rtorrent"
NAME=rtorrent
DAEMON=$NAME
SCRIPTNAME=/etc/init.d/$NAME

checkcnfg() {
    exists=0
    for i in `echo "$PATH" | tr ':' '\n'` ; do
        if [ -f $i/$NAME ] ; then
            exists=1
            break
        fi
    done
    if [ $exists -eq 0 ] ; then
        echo "cannot find rtorrent binary in PATH $PATH" | tee -a "$logfile" >&2
        exit 3
    fi
    if ! [ -r "${config}" ] ; then
        echo "cannot find readable config ${config}. check that it is there and permissions are appropriate" | tee -a "$logfile" >&2
        exit 3
    fi
    session=`getsession "$config"`
    if ! [ -d "${session}" ] ; then
        echo "cannot find readable session directory ${session} from config ${config}. check permissions" | tee -a "$logfile" >&2
        exit 3
    fi
}

d_start() {
  [ -d "${base}" ] && cd "${base}"
  stty stop undef && stty start undef
  su -c "screen -ls | grep -sq "\.${srnname}[[:space:]]" " ${user} || su -c "screen -dm -S ${srnname} 2>&1 1>/dev/null" ${user} | tee -a "$logfile" >&2
  # this works for the screen command, but starting rtorrent below adopts screen session gid
  # even if it is not the screen session we started (e.g. running under an undesirable gid
  #su -c "screen -ls | grep -sq "\.${srnname}[[:space:]]" " ${user} || su -c "sg \"$group\" -c \"screen -fn -dm -S ${srnname} 2>&1 1>/dev/null\"" ${user} | tee -a "$logfile" >&2
  su -c "screen -S "${srnname}" -X screen rtorrent ${options} 2>&1 1>/dev/null" ${user} | tee -a "$logfile" >&2
}

d_stop() {
    session=`getsession "$config"`
    if ! [ -s ${session}/rtorrent.lock ] ; then
        return
    fi
    pid=`cat ${session}/rtorrent.lock | awk -F: '{print($2)}' | sed "s/[^0-9]//g"`
    if ps -A | grep -sq ${pid}.*rtorrent ; then # make sure the pid doesn't belong to another process
        kill -s INT ${pid}
    fi
}

getsession() {
    session=`cat "$1" | grep "^[[:space:]]*session[[:space:]]*=" | sed "s/^[[:space:]]*session[[:space:]]*=[[:space:]]*//" `
    echo $session
}

checkcnfg

case "$1" in
  start)
    echo -n "Starting $DESC: $NAME"
    d_start
    echo "."
    ;;
  stop)
    echo -n "Stopping $DESC: $NAME"
    d_stop
    echo "."
    ;;
  restart|force-reload)
    echo -n "Restarting $DESC: $NAME"
    d_stop
    sleep 1
    d_start
    echo "."
    ;;
  *)
    echo "Usage: $SCRIPTNAME {start|stop|restart|force-reload}" >&2
    exit 1
    ;;
esac

exit 0

Finally we will make the owner of the file the root, make it executable, and have it startup at boot.

sudo chown root:root /etc/init.d/rtorrent
sudo chmod a+x /etc/init.d/rtorrent
cd /etc/init.d
sudo update-rc.d rtorrent defaults

If everything has worked out then running the following should startup rtorrent in a screen session:

sudo /etc/init.d/rtorrent start

Which we will verify by running htop. We should find a few rtorrent processes and a screen process which belong to the user we created earlier. To quit htop hit the “q” key on your keyboard.

The last step is to get rutorrent up and running. rutorrent is just a web interface, and so we will put it in /var/www so that the apache webserver we installed and configured earlier can serve it up to us.

cd /var/www
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/rutorrent

We will also install a bunch of plugins to extend to functionality of rutorrent and to make it more like a native program running on the client machine itself:

cd rutorrent/plugins
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/erasedata
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/create
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/trafic
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/edit
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/retrackers
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/cookies
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/search
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/scheduler
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/autotools
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/datadir
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/tracklabels
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/geoip
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/ratio
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/seedingtime
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/diskspace
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/data
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/rss
sudo svn checkout http://rutorrent.googlecode.com/svn/trunk/plugins/throttle

The diskspace plugin watches the root directory by default, and that is no good since we’re using an external drive for our torrent download location. To fix this we need to edit one line in the file /var/www/rutorrent/conf/config.php:

sudo nano /var/www/rutorrent/conf/config.php

We need to find the line that begins $topDirectory and change it to the following:

$topDirectory = '/media/<mount directory name>';

Finally, we have to change the ownership of these files and folders so that the web server can make use of them:

cd /var/www
sudo chown -R www-data:www-data rutorrent
sudo chmod -R 777 rutorrent

If everything went as expected, then going to https://<local IP Address>/rutorrent should prompt us for a username and password (this is the one we created for the webserver, <webusername>, and is not necessarily the user account we created for the OS). After logging in, we should see the empty rutorrent webgui:

At this point we’re essentially finished with this part of the guide unless you wish to install an FTP server. For my purposes I am content with using sftp, which is built into SSH, though the original source for this portion of the guide also includes instructions for setting up a true FTP server using Pure-FTPd. There are also instructions to enable a multi-user rtorrent setup, though I have not pursued those either since I don’t have any need for it. Since we are finished with rtorrent and rutorrent, you can now start adding your torrents to it. You should make sure to point rutorrent to the right directory  if you add any torrents that you already have the data for and are seeding so that it won’t redownload the entire thing.

*****

Installing and configuring Samba

Samba is a very convenient application to have running for a home media server as it allows us to have very easy access to the files from other computers on the local network. To install samba:

sudo apt-get install samba

We also need to make a samba password for our samba user (the same Ubuntu user we made earlier):

smbpasswd -a <username>

The next step in setting samba up is to change some things in the configuration file, /etc/samba/smb.conf:

sudo nano /etc/samba/smb.conf

The first thing to ensure is that the following line is present and uncommented (it should be below ####### Authentication #######):

security = user

This will ensure that the only people who can access samba shares are those with a valid ubuntu account on the BeagleBone. The next step is to ensure that only people on the local network can access the samba shares which we achieve by adding the following lines under #### Networking ####:

# Added for extra security, only addresses on the local network can connect.
    hosts allow = 127.0.0.1 192.168.1.0/24
    hosts deny = 0.0.0.0/0

You may need to change 192.168.1.0/24 to something else depending on your router and local area network addresses (192.168.0.0/24 is another common one).

The last things we need to do in this config file are to setup the shares themselves. We will comment out any of the lines pertaining to printers because the BeagleBone is not attached to any. The end result is that those lines should appear as follows:

;[printers]
;   comment = All Printers
;   browseable = no
;   path = /var/spool/samba
;   printable = yes
;   guest ok = no
;   read only = yes
;   create mask = 0700

# Windows clients look for this share name as a source of downloadable
# printer drivers
;[print$]
;   comment = Printer Drivers
;   path = /var/lib/samba/printers
;   browseable = yes
;   read only = yes
;   guest ok = no

We also want to enable a share for our attached usb drive by adding the following lines to the end of the file with <mount directory name> and <username> are replaced by the appropriate values we used earlier:

# Share for the external media hard drive
[media]
        comment = External Drive connected to BeagleBone.
        path = /media/<mount directory name>
        read only = no
        browseable = yes
        valid users = <username>

This makes the entire usb hard drive available to the user we created earlier and no one else.

That’s it for samba, and we can enable the changes by restarting the samba processes:

sudo restart smbd
sudo restart nmbd

*****

Updating and maintaining the system

There are a few basic things we need to do to keep our new home media server functioning in tiptop shape, and the first is keeping it up to date. For the most part, this can be done through the ubuntu repositories:

sudo apt-get update
sudo apt-get upgrade

The kernel is a slightly more difficult matter because the one that works with the TI omap processors isn’t upstreamed to the repositories yet. To update the kernel you need to get the one built by Robert C Nelson. Luckily you shouldn’t need to do this very often, except in the case of security patches and important bug fixes. To make this process as easy as possible someone has already created a simple script to do all of the work for you. I have made the necessary changes to configure it to work with the BeagleBone running Ubuntu 12.04.
First we will create a scripts directory to store it, then create the file using nano:

mkdir ~/scripts
cd scripts
nano update-kernel-bone.sh

You then want to paste the following into this file:

############################################################################
## This is a script obtained from http://elinux.org/BeagleBoardUbuntu     ##
## It updates the kernel on the BeagleBone using Robert C Nelsons Sources ##
## https://github.com/RobertCNelson/stable-kernel                         ##
############################################################################

export DIST=precise #(options are lucid/maverick/natty/oneiric/precise/squeeze/wheezy)
export ARCH=armhf   #(options are armel/armhf (armhf only for precise))

#Beagle/Panda
#export BOARD=omap

#BeagleBone
export BOARD=omap-psp

wget http://rcn-ee.net/deb/${DIST}-${ARCH}/LATEST-${BOARD}
wget $(cat ./LATEST-${BOARD} | grep STABLE | awk '{print $3}')
/bin/bash install-me.sh

Using this script is a simple as moving into the scripts directory and running it:

cd ~/scripts
sudo bash update-kernel-bone.sh

Before updating the kernel, it is a good idea to make a backup of the system so it can be restored if something goes amiss. The easiest way I have found of backing up the operating system itself is to use dd to clone the microSD card. To do this you need to remove the microSD card from the BeagleBone after shutting it down:

sudo poweroff

After removing the card you want to insert it into another computer, the one you used to write the card in the first place should work just fine. Running the following command will copy the entire contents of the card to a file called sdcard.img into the home directory of this computer:

sudo dd if=/dev/mmcblk0 of=~/sdcard.img

If there are other SD cards inserted into this computer, then the location /dev/mmcblk0 may be incorrect. Make sure you are copying the contents of the correct SD card before you assume you have a valid backup. You can also change the location the image is copied to some other directory if you wish. This process will take a few minutes to complete, and when it is done you should have an image that can be copied back to the card using the command:

sudo dd if=~/sdcard.img of=/dev/mmcblk0

Hopefully this will never be necessary, though we’re always better off safe than sorry.

If you have another USB hard drive, you may wish to backup the contents of the one on the BeagleBone to it. The easiest way to do this is through rsync using SSH. On your computer (not the BeagleBone, the one we use to ssh into it) you just run the following command with necessary changes to fit your system:

rsync -avv –progress -e ssh <username>@<Local IP Adress>:/media/<mount directory name>/ <the path to the backup drive on the local computer>

The amount of time this takes will depend on the amount of data that is present on the BeagleBone’s drive but not on the destination drive.