Download from any cyberlocker via command line/terminal on Linux or Mac OSX

Screen Shot 2013-05-19 at 19.15.55
Many people are using cyberlockers, like Mediafire, Uploaded, Filepost, … to share files. There are many different providers and all of them keep nagging your with captachs to sell their premium accounts. There are some interesting tools to get around this, like JDownloader or PyLoad.

Those work well if you take the time to set them up and administer them, but might be overkill for people who only occasionally download files from their friends. Here I’m suggesting a simpler command-line based method that won’t require new programs and works with curl or wget. Follow these steps:

  1. Register an account with Zevera.com (yes, you need to pay a little bit, but your account is based on actual usage and never expires)
  2. Add the following lines to your .bashrc or .profile (when on a Mac). This adds a new function to your command line environment. The only input-parameter it accepts is the URL or the file you want to download.
    zeveraDownload() {
            curl --proxy-anyauth -p -C - -c ~/.zeveraCookie.txt -u USER:PASSWORD -L https://zevera.com/getFiles.ashx?ourl="${1}" -J -O
    }
  3. Open a new terminal-window and simply enter
    $zeveraDownload http://ul.to/xxxx

After following a few redirections, your download should start. Curl also uses the correct filename. Please be aware that this technique will only work with supported hosters and might not always be reliable. I still worked well for me so far. The alternative is setting up pyLoad and signing up for an Indian captcha-solving-service.

Scan image via SSH

We have an ancient Brother scan+print combination in our apartment. Cheap and reliable. Printing works well over CUPS, also from iPads with Airprint. My only issue was scanning. It works well with SANE, which also works via your network. Only problem is that the packages provided by Mattias Ellert on his website are a bit outdated and don’t always work with the latest Mac OSX. So here is a quick trick to scan on a remote machine without the trouble of moving the file later:

 ssh cyrus@gaia.local "sudo scanimage --resolution 400 --format=tiff" | convert tiff:- -quality 90 -quiet `date "+%Y-%m-%d_%H-%M-%S"`.jpg  

Basically this command pipes the image over SSH and hands it over to the local convert for compression. Neat and elegant. :-)

XBian – Raspberry Pi-based distro for running XBMC

Here just some upaid advertising for a nice armhf-based XBMC distro, I discovered some time ago. If you just want to use your RPI for watching movies and TV-shows, this is definitely for you. It auto-starts XBMC, mounts USB-drives and lets you connect to NFS and SMB as well. They only thing you might want to do is keep your XMBC-user folder on a different drive, so you can easily switch versions and always have a backup.

Link: http://xbian.org/

Raspberry Pi for CCTV using motion

I wrote about doing CCTV with zoneminder before, but realized that this might be overkill for many people. If you use less than 4 cameras, you can use the motion package. It works nicely with the Logitech webcam, I connected to my Raspberry.

The camera worked out of the box except that saturation, brightness, etc was way off. You can configure it with the v4l2-utils package. Once you have verified that the picture is OK by using uvccapture, you should be ready to install motion. For my USB webcam, motion worked out of the box. No settings needed at all. For finetuning, I changed the framerate in /etc/motion/motion.conf to 25. Even at that rate, load stayed at 0.00 with abotu 20% CPU usage. Maybe the GPU is doing the bulk of the image processing work.

By default motion captures .swf and .jpg. For me .swf is enough, so I disabled .jpg output with output_normal off.  If you want to view your cam over a browser, set webcam_localhost off. That way every IP address can access the video stream over a local address like http://10.0.0.19:8081. Further up, you can see my stream. Unfortunately brightness is still a bit too low. I’ll change that later.

Now comes the part where we make our setup really useful. By default motion saves all captures .swf files in the current working directory. When using the init-script, you should save a sensible default output directory. You can change that with the target_dir parameter in /etc/motion/thread1.conf or motion.conf.

So far so good. By now we have motion running and saving all observations to a directory. That’s OK, as long as we can just SFTP into the raspberry and view the footage. If you want your mum to use it, there needs to be something simpler. I used the PHP-script MotionBrowser by Carlos Ladeira as a simple way to view events by day and hour.

For the script to run, you need to install a web server with PHP-support. Apache is overkill here, so I recommend lighttpd or nginx. I won’t go into the details of how to install a webserver, but believe there are many tutorials out there. On top of that you also need mysql to manage the event list. The setup is described on the MotionBrowser site.

Once your webserver is running download motionbrowser.tar and unpack it into /var/www or any other web server root dir. The script worked well, except for serving the .swf-files. When checking download.php, you will notice that the script is actually made to serve .avi-files. We could change that in the motion-settings, but I prefer to stay at .swf because they can be viewed in the browser. Here is what needs to be changed:

In line 53 change avi to swf, so only .swf-files can be downloaded.

1
 if (strtolower(substr(strrchr($filename,"."),1)) != "swf") return;

And also replace the whole header part with just one line to simply view the SWF.

1
 header('Content-Type: application/x-Shockwave-Flash');

That should be it. Enjoy your USD 35 surveillance system.

Update:

If you don’t like refreshing your browser all the time, you can view the live stream in vlc as well.

Using Raspberry Pi for CCTV with Zoneminder

Note: zoneminder might be overkill for many people. If you have less than 4 cameras, also read my article about doing CCTV with the motion package.

One of the first uses, for the Raspberry Pi, I thought of was qick-and-dirty CCTV. This could be useful for scenarios, like remote construction sites, senior citizens, holiday homes, etc.

The only gear needed is a Raspberry Pi, a USB webcam and internet access through a router. The webcam could also be a network camera or a network digitizer. That way you have more flexibility with your location. The router could be substituted for a 3G data stick. If you have a spare TV, you could also use it for viewing those cameras. The steps for setup are:

  1. Connect router, USB-camera and power to your Raspberry.
  2. Flash my favourite SD-card image on a card. Steps are describe on the website. Be sure to enable SSH-access. Or connect a TV via HDMI.
  3. Install the zoneminder package and dependencies with apt-get install zoneminder
  4. Make sure zoneminder and its related services are working with service zoneminder restart
  5. Now you should be able to access the zoneminder interface via http://raspberry-pi/zm
  6. If you got that far, you could already add IP network cameras to your interface. There are some really cheap Chinese models to be found on ebay for about 35 EUR. Just make sure they provide you the image over a fixed path like 10.0.0.22/image.jpg. If they only come with an ActiveX web interface, they are pretty much worthless.
  7. To include a USB camera, make sure it is supported by the Linux kernel. You can check connected devices with lsusb and dmesg after plugging it in. If everything is working, your camera should show up with zoneminder as local interface.
  8. If you want to view your camera on your phone, there are a number of apps available for Android and iPhone/iPad. EyeZM worked particularly well and also allows you to view recorded events. The only thing you need is an external domain with some kind of dynamic DNS and port forwarding on your router. You could use DD-WRT to do both jobs. Or install a cron-job on the Raspberry to update its external IP regularly. A flexible and open DNS-service is ClouDNS. Try to avoid DynDNS. They are way to expensive for what they offer and use questionable tactics to buy out competitors.

These are the rough steps to turn your Raspberry Pi into a surveillance machine. Since the machine is not particularly high powered, don’t connect more than 2 or 3 cameras at a time. Motion detection and recording will also put some pressure on it. There are some tricks to install optimized JPEG-libaries. This is something you can try.

If you already have analogue cameras, I recommend the Aviosys IP9100. It costs around 100 EUR and can digitze up to 4 analogue cameras. To pull separate streams, install this alternative firmware. It allows you to access each channel through /usr/yoics[0-3].jpg

I hope this article gave you some inspirations and ideas of what can be done. Be warned that zoneminder is not the easiest beast to configure and install. I will try to come up with a setup script or a pre-built SD-card image at some point.

Access your Raspberry-Pi through any Browser with VNC

The other day I set up Ubuntu in a virtual Machine to do some preparation work before deploying a similar setup in Amazon’s EC2. This basically included programming some Java in Eclipse. When I had to show some stuff to a colleage, I found a quick trick to share a screen remotely on Lifehacker.

This also works well, if you don’t have a screen for your Raspberry or want to access some work files, while on the road. First you need to install the necessary packages

1
sudo apt-get install x11vnc vnc-java

Then just set a password. Username is not required.

1
x11vnc -storepasswd

Now start VNC with

1
x11vnc -forever -usepw -httpdir /usr/share/vnc-java/ -httpport 5800

Now you can try accessing your Raspberry Pi on the local network on http://raspberry-pi:5800. If you want outside access, you could either forward the necessary ports or maybe use IPv6, if already supported from your provider.

The port for plain vanilla VNC is 5400 and works with the same password. For Mac OSX, Chicken of the VNC is a good client. For extra security, you should add SSL to the VNC-connection.

Use Raspberry Pi to Print From Your iPad or iPhone

Yesterday, I described how you can use your Raspberry Pi to download and access torrents. Today my parents asked me to help them set up printing via their iPads. They have a rather old HP Deskjet 5940 printer with a simple USB interface.

First you need to install CUPS (common unix printing system) for printer access. In my case the required printer drivers were installed automatically (hplip). If you are using a different printer you might have to check which packages you need.

After cups is installed (took about 20 min here), enable remote access to it by editing the file /etc/cups/cupsd.conf in 3 places. Add those lines in the relevant places:

1
2
3
4
5
Listen *:631
Order allow,deny
Allow 10.0.0.*
Order allow,deny
Allow 10.0.0.*

After editing the file and restarting cups with service cups restart you can login to your print server on http://raspberry-pi.local:631. Simply click Administration > Add Printer and follow the steps. If the correct driver package is installed, and the printer is turned on, it should show up on the top of the list.

If your printer installed correctly, it show up in your list of nearby printers, as well as on your iPad and iPhone. It’s not necessary to produce a special avahi-script any more, as with older iOS versions.

Raspberry Pi as Budget BitTorrent Server

The raspberry pi is a surprisingly powerful and cost-efficient computing device. Yesterday I put together this simple setup script to turn your new raspberry into a mean green downloading machine. The only requirement is an existing internet router and a working raspberry with SD-card.

The guys on the official raspberry site have some nice images for downloading, but since we won’t be using the graphical UI, we can use a the slim image put together by darkbasic on his site linuxsystems.it. Just follow his instructions to download, extract and install the image to a compatible SD-card. A list of tested and compatible SD-cards is available from the official site.

While installing make sure to enable SSH-access by renaming the boot-enable-ssh.rc boot.rc

Once your raspberry has booted, you need to find out its IP by looking at the router’s DHCP table. Just look for a device name raspberry-pi.

Next just create a new text file by typing vi setup.sh in your terminal window while logged into the RP. Then just copy the script provided with this blog post to your clipboard. Press “i” to go into editing mode and paste the script with Ctrl/Cmd+v. Then save it with ESC, : x. Execute the script by issuing sh setup.sh. As opposed to the default Debian image, the darkbasic image gives you root access by default. When using another image, execute the script with sudo sh setup.sh.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
#!/usr/bin/env bash

# Installs all components needed for a nice seedbox on your new raspberry pi.
# Base image: http://www.linuxsystems.it/2012/06/debian-wheezy-raspberry-pi-minimal-image/
# Prepared by manu (at) snapdragon.cc

echo "[+] Updating packages"
apt-get update ; apt-get -y upgrade

echo "[+] Installing Samba and Transmission packages"
apt-get -y install transmission-daemon samba avahi-daemon

echo "[+] Configuring Transmission"
service transmission-daemon stop
sed -i -re 's/(rpc-authentication-required\":\ )([a-z]+)*/\1false/g' /etc/transmission-daemon/settings.json
sed -i -re 's/(rpc-whitelist-enabled\":\ )([a-z]+)*/\1false/g' /etc/transmission-daemon/settings.json
sed -i -re 's/(download-dir\":\ )([a-z]+)*(.*)/\1\"\/srv\/media\",/g' /etc/transmission-daemon/settings.json
service transmission-daemon start

echo "[+] Setting up public Samba share"
mkdir /srv/media
chown debian-transmission /srv/media

echo "[public]
comment = Public Shares
browsable = yes
path = /srv/media
public = yes
writable = yes
guest ok = yes"
>> /etc/samba/smb.conf
service samba restart

Now just sit back and wait for all components to install. Depending on the speed of your SD-card, this can take up to an hour. The script will first update your package manager, then install the required applications and dependencies and settings. If you’re using a different image, you might need to adapt some package names.

If all went well, you should be able to access the transmission web interface via your Mac on the following URL: http://raspberry-pi.local:9091. On Microsoft Windows, you can try http://raspberry-pi:9091. In case neither name resolution should work, just use the IP address, you found out via your router before. The inteface should always be available on http://<raspberry pi IP>:9091.

On the transmission web interace, you can add and manage all your (legal) torrent downloads. Depending on the size of your SD-card you might need to delete old downloads from time to time.

To access your downloads, you can mount it via CIFS/SMB. On your Mac go to the Finder and connect to a new network folder by pressing CMD+k. Then type smb://raspberry-pi.local, choose to connect as guest and you should see the right folder. On Windows you can enter \\YOUR RASPBERRY IP\public in Windows Explorer. If you are a heavy Windows user, you can also enable WINS address resolution in smb.conf

If everything went well, you should now have a basic BitTorrent server. In case something doesn’t work, just start over with a new image and do the steps found in the setup script one by one. If your downloads work OK, but you can’t access them there might be a permissions issue. This is easily correctable by logging in doing a chmod -R 777 /srv/media.

Please note that you can only use this setup in a trusted network, because all user authentication and security mechanisms are disabled. If you want to access your torrent server from the public internet, you need to proxy it via a proper webserver, enable HTTPS and password authentication.

Since the whole system is running on a standard Debian image, you could add all kinds of cool features like:

  • adding torrents automatically with Flexget
  • downloading from cyberlockers, like Rapidshare with PyLoad
  • share files via HTTP with Apache and h5ai

Easily Resize PDF on Mac OSX and Linux

On many occasions, you might want to resize a PDF to send it by Email or put it on the web. There are many useless Share- and Bloatware-tools (like Adobe Acrobat) that might or might not help you with that.

If you happen to work on a Linux, BSD or Mac OSX machine, you can simply use ghostscript for the task. It’s much faster and gets better filesizes than the Optimize-function of Adobe Acrobat.

All you need to do is to make sure ghostscript is installed and then put the following shortcut in your .profile or .bashrc in your home-directory.

pdfResize() {
gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dNOPAUSE -dQUIET -dBATCH -dPDFSETTINGS=/$1  -sOutputFile=output.pdf $2
}

After that’s done, open a new Terminal-window and try the following command:

pdfResize screen MyDocument.pdf

You should now find the resized document in a file called output.pdf in the same directory.

The first argument sets the quality. Available options are screen, ebook and printer. While screen gives you the smalles file size, printer provides better quality at a reasonable filesize.