Thursday, October 30, 2014

Nerdy Christmas Gift

Continuing the theme from the last post, here is a nerdy Christmas gift my awesome wife got me several years ago.  A set of drinking glasses etched with the writing "#include <milk.h>"  If this means nothing to you, then you can read this and this.


And yes, my favorite beverage is milk!

(Sorry, because of the curvature of the glass I can't get a single picture showing the whole thing)

Coolest Birthday Gift

Earlier this week was my birthday, and my wife got me the coolest gift a nerd could ask for.  A Lego Nintendo Entertainment System (NES).  This is not an official set, but comes from the creative mind of Chris McVeigh.  As someone who grew up in the '80s with Legos and video games, this was the perfect gift that combines those two.









Thursday, October 23, 2014

Raspberry Pi DVR - Addendum

I recently completed my Raspberry Pi DVR and put it into regular use.  But I quickly realized something was wrong.  Some recordings would work, but most recordings would fail 10, 15, 20 minutes in.  Every time it failed I noticed the same thing, if I SSH into the Pi and looked at the running processes, tvheadend was no longer running.  The mere fact that I can SSH in to the Pi means that the Pi itself and Linux were still running.  So what is happening, is tvheadend crashing?

The Problem
I started with the assumption that tvheadend was crashing, after all that seems the most logical explanation.  But where do I go from there?  I researched Linux logging and diagnostics to see if the system recorded more info about this problem.  Eventually I stumbled across /var/log/kern.log which is the log file for the Linux kernel.  Every time there was a failed recording, the following was logged:
Oct 20 20:10:00 garagepi kernel: [79819.240765] Out of memory: Kill process 2314 (tvheadend) score 922 or sacrifice child
Oct 20 20:10:00 garagepi kernel: [79819.240785] Killed process 2314 (tvheadend) total-vm:607540kB, anon-rss:404384kB, file-rss:384kB

Aha!  So the problem here is not that tvheadend is crashing, the problem is Linux ran out of memory so to keep the system up and running, it killed the process using the most RAM which was tvheadend.

Because the Raspberry Pi only has a paltry 512MB of RAM, tvheadend is able to consume that and more resulting in this problem.  But if we dig deeper, what is going on here?  Does tvheadend have poor memory management?  Does tvheadend have memory leaks?  Is the task of recording live TV just too much for the limited resources of Raspberry Pi?  Or is there something else going on here?  I think the answer is mostly in the "something else going on here" category.

Let us start off by looking at what tvheadend is doing.  Basically all it is doing is shuttling data from one pipe to another.  The "input" pipe is the digital TV stream coming in from the TV tuner.  For ATSC TV, this amounts to approximately 5GB of data per hour.  That data is then written (unmodified) to the "output" file.  In my setup, the output file is located on my NAS box, so the data must be transmitted over wifi to get there.  And therein lies the problem.  What if the Raspberry Pi cannot transmit that 5GB/hour of data over wifi fast enough?  Linux, like all modern operating systems, would allocate more buffers to hold that data until such time that it is written over wifi.  This means if wifi is too slow, tvheadend will continue to consume more and more system memory in the form of output buffers until such time as Linux steps in and shuts it down.


The Solution
Now that we know the problem, what can we do about it?  The short answer is, we need to improve the memory "situation" of the Raspberry Pi.  For this I made 3 major changes.

  1. The Raspberry Pi has 512MB of RAM, which is split between both the CPU and GPU.  By default the GPU takes 64MB of RAM.  Since this is a headless unit in my garage, let us change that so the GPU gets the bare minimum.  Editing the config.txt file, set "gpu_mem_512=16"  This gives 16MB to the GPU which is the minimum.  This frees up 48MB of RAM for the CPU.  One downside is the GPU requires more memory than 16 for HDMI output.  I have read that either 24 or 32 is the minimum required for HDMI output.  So if you want HDMI output then set this value accordingly.  Either way, this setting is the single largest way to free up memory.  No other step will come close to the 48MB this frees up.
  2. Second step is to stop any unnecessary background processes.  I am no Linux expert, so it was hard for me to know what to stop and how to stop it.  But doing some research, everyone suggests turning off the getty processes.  To do this edit the file /etc/inittab.  Scroll down and look for the "getty" lines.  Disable the getty lines by placing a '#' in front of that line.  Reboot and they are not disabled.
  3. The final step is to do anything possible to improve wifi speeds.  Faster wifi means fewer buffers.  On Raspbian there are at least 2 ways to check your wifi connection details.  The first is "iwconfig wlan0" and the second is "cat /proc/net/wireless"  Both show two numbers, the link quality and signal level - both measured 0 to 100.  Both numbers are important.  What you want to do is adjust the position of your wifi antenna to increase these numbers.  My starting signal level was low 50s, after adjusting the antenna I was able to reach the high 80s.  Just that little bit made a huge difference!  Timing the copy of a large file across wifi, with signal in the 50s it took 1 minute and 29 seconds.  With signal in the 80s that dropped to 4 seconds!  So without a doubt, wifi antenna placement is critical.  Before you start fine tuning your wifi, run the command "watch -n 1 cat /proc/net/wireless"  This will display the current wifi settings updated once a second.  This makes it easy to see signal levels as you move the antenna.



Conclusions
If you wish to undertake a project like this and turn your Raspberry Pi into a DVR, there are several things that I consider crucial to your success.

First, make the above changes to give the minimum memory to the GPU and shutdown any unnecessary processes.

Second, if at all possible, use the ethernet jack instead of wifi.  Wired is always going to be faster and more reliable.  So only use wifi if you have no other option.

Third, if you do use wifi be sure to turn your wifi for maximum signal strength and speed.  Consider one of those homemade wifi antenna reflectors (plans available on the Internet) to improve wifi signal.

Fourth, with wifi optimized if you still run into problems, consider recording the video locally.  You can record the video directly to the SD card or onto a USB thumb drive or external hard drive plugged into the USB port.  Whereas it is not ideal to record large files directly to flash memory, you may have no choice.  After recording you can always copy the file over wifi during idle time when it is no longer a time sensitive process.

Fifth, consider frequent reboots.  Linux is supposed to be stable over the long run, but this does not mean all daemons (like tvheadend) are.  I think I am going to set up a cron job to reboot the Raspberry Pi every night.  After all, it is not doing anything else at 3 in the morning, so why not reboot and help tvheadend to free up memory?

Tuesday, October 21, 2014

Raspberry Pi DVR - tvheadend

Now that the final configuration of the Raspberry Pi is complete, it is time to install and configure tvheadend.  I felt this process was complex enough to warrant a dedicated post.

Installation:
Installing applications on most Linux distros is very straight forward, you issue a command which downloads and installs the package from the repositories on the Internet.  Unfortunately, the tvheadend package is not a part of the Raspbian repositories, so you can either download the source code and compile it yourself or you can manually connect to another repository.  I went with the latter, specifically using the repository maintained by tvheadend themselves.

First run the command "sudo curl http://apt.tvheadend.org/repo.gpg.key | sudo apt-key add -"  This adds the tvheadend GPG signing key to your system.  Next run "sudo nano /etc/apt/sources.list"  Append "deb http://apt.tvheadend.org/stable wheezy main" to this file.  Lastly, run "apt-get update" and "apt-get install tvheadend"  This will install and start the tvheadend server.  Installation will prompt you for a user account.  You can use the default Raspbian login, or use another account you previously created.

Configuration:
Now that tvheadend is installed we can configure it.  I wish I could say that tvheadend is easy to configure... but it is not.  I spent a lot of time learning the ins and outs of tvheadend before I was finally able to get it to work.

First you need to open a web browser and navigate to "<server>:9981" where <server> is the IP address of your Raspberry Pi or the name (if you enabled Samba).  You can use Epiphany from the Pi itself, or browser on another machine.

You may be prompted for a username and password, if so enter the credentials above used during the installation of tvheadend.  Once tvheadend is displayed, click on Configuration | DVB Inputs | TV Adapters.  From the combo box select your TV tuner.

If you do not see your TV tuner in the combo box then chances are you forgot to download the firmware file into /lib/firmware.  Consult the Linux TV Wiki for more info.

Make sure the device is enabled (checkbox in the middle) and click Save.  Next click on the "Add DVB Network by location" button.  Select the TV signal you wish to add.  For digital over-the-air broadcast TV in the U.S. select "us_ATSC_center_frequencies_8VSB."

After doing this, the right side of the screen where is says "Muxes awaiting initial scan" should read something like 68.  This will slowly countdown to 0.  Although there is little feedback, tvheadend is scanning all possible channels looking for signals.  This will take time - 10, 15 minutes, or more.  When that number reaches 0, hopefully the "services" and "muxes" above will read something other than 0.  That means it found some channels.  If "services" and "muxes" still read 0, then you need to check connections.  Is the antenna connected to the TV tuner?  If the antenna in a good location?

Once you have found services and muxes you are ready to map to channels.  Here is where things get really painful.  You can click the "Map DVB services to channels" button, but in my experience this does nothing.  Instead, we need to manually map these channels.  Here is how.  First, click on Configuration | Channel/EPG | Channels.  Next click "Add channel" and give the channel a name, something like "NBC 6-1."  You should also give it an integer number, but as far as I can tell this number means nothing.  Be sure and click "Save changes" after adding the channels.  Here is a screenshot showing manual entries for all the channels in my area.

Now that you have manually created the channels, you need to manually map them to the services found in the prior scan.  Click on Configuration | DVB Inputs | Services.  All found services should be listed, but the Channel name column should be blank.  For each and every row, double-click on the Channel name column.  This should open up a combo box and allow you to select one of the channels you manually created.  But how do you know which channel to select?  You basically need to do a double-mapping, manually of course (welcome to the world of Linux, do as much of the leg work yourself).  Every digital broadcast channel in the U.S. has something called a physical and virtual channel.  The virtual channel is "6.1" but the physical channel might be 15.  You can use antennaweb.org or tvfool.com (or probably a host of other sites) to get physical and virtual channels for all stations in your area.  The second mapping is physical channel to frequency.  A simple google search will show frequencies in MHz for all physical channels.  Now you are ready to map channels.  For each row there is a Multiplex column.  This shows the frequency for the channel.  Map this frequency to the physical channel, then map that physical channel to the virtual channel, then select that channel in the Channel name field.  After making all these changes, do not forget to click Save changes.  If this step is too complex, just use trial and error.  Set a channel to something, anything, then watch that channel.  Once you know what it really is, go back and change it.

You are getting very close now to completion.  Next click on Configuration | DVB Inputs | TV Adapters.  From the combo box select your TV tuner.  In the settings in the middle, make the following changes, be sure and click Save when you are done.

  • Enable "Skip initial scan."  This will prevent tvheadend from scanning for new channels every time the system reboots.
  • Disable "Idle scanning."  This will prevent tvheadend from continuously scanning in the background for new channels.
  • Enable "Close device handle when idle."  Without this tvheadend keeps the TV tuner active 24/7 which consumes a lot of electricity plus the TV will get really hot.  With this enabled the TV tuner is put into standby after each use.

The final piece of configuration can be found by clicking on Configuration | Recording | Digital Video Recorder.  Make the following changes, and be sure to click Save configuration when done.
  • For the recording path I entered "/mnt/nas/Videos"  This is where I want recorded videos to live, over the network on my NAS box.
  • Change Media container to "Same as source (passthrough)."  This saves the broadcast stream to a video file without any changes (called transcoding).  Since the Raspberry Pi does not have a powerful processor, it is unlikely it could transcode on the fly in real time.
  • I also checked boxes at the bottom control the output filename.


There, that was not so bad, was it?  Ok, I admit, that was a major pain in the butt!  Getting tvheadend is a pain to setup, but once it is done, it works pretty well.  Again, welcome to the world of Linux, expect more effort on your part, and less user-friendly compared to a typical Windows application.

Now that tvheadend is configured, you can set it to record TV programs.  To do that login to the above tvheadend configuration page.  Click on Digital Video Recorder | Upcoming Recordings.  Click Add entry and select channel, date, and time for the recording.  This is just like programming an old VCR.  You tell tvheadend to record channel 6.1 from 8:00 to 9:00 for example.  It uses 24-hour notation, so 5pm is represented as 17:00.


One area I would like to improve here is the use of EPG (electronic program guide).  This is a digital TV guide showing you what is on each channel at a given time.  You can simply click on a show to record it.  Or tell tvheadend to do things like "record the Simpsons" and it will records all episodes of the Simpsons, regardless of channel or time.  I have not configured EPG, for now I am doing everything manually.  EPG would be nice, but frankly after all this, I am just glad it is working.  So maybe in the future I will decide to tackle EPG.

Raspberry Pi DVR - The final configuration

I have finally reached the end of my Raspberry Pi DVR project.  My goal was to turn a Raspberry Pi into a DVR to record over-the-air broadcast TV.  To that end I have spent the last month experimenting with all the major distros available for the Raspberry Pi in an effort to determine the best distro.  Below is a step by step summary of my final configuration.

I ended up using Raspbain Wheezy as the operating system.  I downloaded the latest image (September 2014) direct from the Raspberry Pi Foundation's website.

Download the Win32 Disk Imager on your Windows computer.  Use this tool to write the above Raspbian image onto your SD card.

Stick the SD card into the Raspberry Pi and plug in power.  During first boot the Raspberry Pi configuration screen will come up.  I made the following changes.

  • Select "Internationalization Options" and then "Change locale."  Place a check by "en_US.UTF-8 UTF-8."
  • Select "Internationalization Options" and then "Change time zone."  Linux handles time zones differently than Windows.  Instead of selecting your time zone by name or UTC offset, you instead select a region and a city near you.  So for me I select Americas and Los Angeles.
  • Select "Internationalization Options" and then "Change keyboard layout."  For me I selected "Generic 104 key" and "English US."  This step is important as the default keyboard layout is English UK with things like the pound sterling symbol.
  • Select "Advanced Options" and then "Overscan."  Set this option to disabled.  Older analog TVs have overscan where the image is smaller than what the TV can display.  If I do not disable overscan then on my computer monitor I have half an inch of black all the way around the outside edge.  Disabling overscan corrects this.

Note: I did NOT set the Raspberry Pi to autoload the GUI (X desktop) on startup.  Because this will be a headless unit, having X running is just wasted resources.

Now would be a good time to configure the Raspberry Pi "BIOS."  Shutdown Linux, remove the SD card, and stick it into a Windows machine.  Append the following lines to config.txt:
arm_freq=800
core_freq=300
force_turbo=1
max_usb_current=1
The first 3 lines add a little overclocking which is totally safe, but definitely helps out.  The last line is required for the model B+ to take full advantage of the new USB power circuitry.  By default, the model B+ will deliver up to 600mA of power to the 4 USB ports.  However, with the above setting it can deliver up to 1,200mA (assuming you are using a good quality 2A power supply).  600mA might be enough under normal uses, but with the TV tuner and wifi card on top of keyboard and mouse, you will need the extra amps.

Next I need to configure my wifi.  I manually started the GUI desktop with the command "startx"  I used the shortcut on the desktop to configure my wifi connection.  There are ways to do this from the console, but it was just easier to do it graphically.

Now that the network is setup, we can configure the Raspberry Pi on my network.  By default Linux does not play well with other computers on the network.  Access to and from the Pi works only when using IP address and not computer names.  To fix that we need to install a networking package called Samba.  To install Samba run the command "sudo apt-get install samba samba-common-bin"

The next step is to give your Pi a computer name.

  • Run the command "sudo nano /etc/hosts"  There should be a line with "127.0.0.1   raspberrypi"  Change the "raspberrypi" to whatever name you wish to use.
  • Run the command "sudo nano /etc/hostname"  This should be a single line text file.  Replace the text with the name you entered above.
  • Commit the above changes with the command "sudo /etc/init.d/hostname.sh"  [I am not sure if this step is required.]
  • Reboot your Raspberry Pi.  After the reboot it should have a new computer name and you should be able to access computers via name.


In my setup, I want the Raspberry Pi to record TV programs directly to my NAS box.  To do that I need to mount the NAS box so that the Raspberry Pi can access it.  First, change to the /mnt folder.  Run the command "sudo mkdir nas"  This will create a folder /mnt/nas.  This folder will become the mount point for the NAS drive.  Now run the command "sudo nano /etc/fstab"  Scroll to the bottom and enter the following:

//<server>/<share> /mnt/nas cifs username=<user>,password=<pwd> 0 0

Where <server> is the name or IP of the NAS box and <share> is a shared folder where to save the videos.  Also <user> and <pwd> are the credentials used to access the NAS box.  After rebooting the Pi again, you should be able to browse to the NAS box at /mnt/nas.

To get my Hauppauge 950Q TV tuner working, I need to manually download the firmware file.  I started the X desktop again and used the Epiphany web browser.  I downloaded http://www.kernellabs.com/firmware/xc5000/dvb-fe-xc5000-1.6.114.fw.  I then moved the file to /lib/firmware

After all this work, it is finally time to install and configure tvheadend, which is the software that performs the actual TV recording and streaming.  This step is not trival, so I will cover everything in a dedicated post.

The final step, once all this is configured, is to ensure the Raspberry Pi is up to date.  Run the command "sudo apt-get update" which checks for available updates (but does not install them).  Then run the command "sudo apt-get dist-upgrade" to download and install the updates.  I have found that the second command often times hangs toward the end.  If this happens, unplug the Pi and plug it back in.  Once it comes back up rerun these two commands.  If they succeed then you are done.  However, they might fail and tell you to run another command first.  Run the command it tells you to run and that should clear up any problems.

There you have it.  An end to end configuration guide for turning a Raspberry Pi into a DVR for TV recording.  I hope you found this guide helpful!


Update: This addendum covers changes I made to help with tvheadend crash problems.