Wednesday, June 10, 2009

How to Bulk Download YouTube Videos on Linux and Convert to MPG and MP3


This tutorial shows how to bulk download YouTube videos on Linux and then convert the FLV files to MPG video and MP3 audio files.
Required Software

I am using Ubuntu Linux, but the concept should be the same for other distros. Make sure you already have all of the multimedia codecs including ffmpeg and mplayer. You will also need to install youtube-dl:

sudo apt-get install youtube-dl
Downloading YouTube Videos on Linux

To download YouTube videos on Linux, use youtube-dl in the terminal like this:

youtube-dl [options] video_url

To see available options type man youtube-dl in a terminal.
Converting YouTube Videos to MPG on Linux

To convert YouTube movies from FLV to MPG on Linux use ffmpeg like this in the terminal:

ffmpeg -i old-file.flv new-file.mpg
Ripping MP3s from YouTube Videos on Linux

It is easy to extract MP3 audio from a YouTube movie on Linux with mplayer.

The method to rip MP3 audio from FLV files with ffmpeg from this site doesn't work on my computer so I use mplayer to do it:

mplayer -dumpaudio old-file.flv -dumpfile new-file.mp3
Automating the Process with a Shell Script

If you want to download a set of YouTube movies and convert them to MP3 audio, or save them as MPG movies on your hard drive, you can automate the process with a shell script.

This is just a quick script showing how it might be done. Use the following code at your own risk, and make sure you understand what it does before doing it. It will delete all the FLV files in the current directory after it processes them.

#!/bin/sh

# This script downloads YouTube videos and converts them
# You will end up with an MPG video and MP3 audio of each YouTube movie
# You need to install the following three programs:
# youtube_dl, ffmpeg, and mplayer

# Use this script at your own risk!
# Make sure that you understand how it works before you use it!

# USAGE:
# Make a file called videos.txt with the URL of 1 YouTube video on each line.
# Don't leave any blank lines in the file
# Put the videos.txt file in an empty folder along with this script
# run this script with the following commands:
# $ ./youtube_downloader.sh
# It will take a long time to run, depending on
# how many videos you have in your videos.txt file.

while read inputline
do
youtube_url="$(echo $inputline)"
youtube-dl $youtube_url
done < videos.txt

# Convert the flv files to mpg and mp3 files
for i in *.flv
do
ffmpeg -i $i $i.mpg
# comment out the next line if you don't want MP3's extracted
mplayer -dumpaudio $i -dumpfile $i.mp3
# The next line deletes all FLV files in the current directory to cleanup
rm $i
done

See also pytube...

Thinking out of the BoX !! Download YouTube Videos : The Linux Way !


In this article I will outline a few methods to download YouTube Videos to your system and getting them to play in your favorite player. We will go in a step-wise way :

A. Getting the Video

1. Online Tools

There are some sites like KeepVid.com which allow users to input a YouTube link and they would give you the download link. So get the download link from this site and download the flv file.
2. Firefox Plugin
There is a firefox extension which allows you to download YouTube videos as you watch them. I haven’t tried this.
3. Command Line Tools
This is the safest and best way (for CLI junkies :-D ). Download the script YouTube-dl (thanks to Ricardo Garcia Gonzalez for this nice script). Just put that script in your ~/bin or /usr/bin directories or somewhere in the application path.
To download the flv video do this :
youtube-dl
e.g : youtube-dl http://www.youtube.com/watch?v=D1R-jKKp3NA

B. Playing the video

1. Use VLC Player or MPlayer or Kaffeine to play the flash video.
2. If the above method doesn’t work out (It din’t in my case)..Then go the difficult way :
* Get ffmpeg for your system. SuSE users can get it here.
* Convert the flv video to avi (or any other format) using ffmpeg :
ffmpeg -i my_flash_video.flv the_avi_file.avi
* Now drag and drop the avi file into your favorite video player n watch the video :-)

Still unsatisfied ?! “Dude I’m a *nix user. I can’t sit and do all these steps to get a simple video playing !!” No probs. We will write a shell script to do all this work for you.

Note : I found the downloader in youtube-dl quite slow. So I tweaked it a bit(I don’t know python :-D ) to use an external downloader (axel rocks!). Open the file youtube-dl in your favorite editor and change the following lines :

282 # Abort here if in simulate mode
283 if cmdl_opts.simulate:
284 sys.exit()

to the following :

282 # Abort here if in simulate mode
283 if cmdl_opts.simulate:
284 cond_print(video_data.geturl())
285 sys.exit()

That will make the script output the downloadable video link to the standard output (when run in simulate mode). Now write the shell script :
#!/bin/bash
if [ $# -ne 2 ]
then
echo "Usage : $0 "
echo "e.g : $0 http://www.youtube.com/watch?v=D1R-jKKp3NA steve_jobs"
else
outputfile=".avi"
todnload=`youtube-dl -s $1 | tail -n 1 -`
echo "Got the file.."$todnload
axel -n 100 $todnload -o $2".flv" # wget -c $todnload -O $2".flv" .if you don't use axel
echo "Download Completed..."
ffmpeg -i $2".flv" $2$outputfile # get the avi file
fi

Done. Now paste that code into a file utube in your path. Then run utube as :
utube http://www.youtube.com/watch?v=D1R-jKKp3NA steve_jobs

This will give 2 files steve_jobs.flv and steve_jobs.avi ;)
Just drag the *.avi to kaffeine and enjoy the speech by Steve Jobs !

Monday, June 8, 2009

Arrow keys not working in VMware Server 64bits, host=Fedora 11 64bits, Guest = XP pro 32bits


Ok, I got the solution:

Edit your "/etc/vmware/config" file, so type:
sudo gedit /etc/vmware/config


And add this to it:
xkeymap.nokeycodeMap = true
xkeymap.keycode.108 = 0x138 # Alt_R
xkeymap.keycode.106 = 0x135 # KP_Divide
xkeymap.keycode.104 = 0x11c # KP_Enter
xkeymap.keycode.111 = 0x148 # Up
xkeymap.keycode.116 = 0x150 # Down
xkeymap.keycode.113 = 0x14b # Left
xkeymap.keycode.114 = 0x14d # Right
xkeymap.keycode.105 = 0x11d # Control_R
xkeymap.keycode.118 = 0x152 # Insert
xkeymap.keycode.119 = 0x153 # Delete
xkeymap.keycode.110 = 0x147 # Home
xkeymap.keycode.115 = 0x14f # End
xkeymap.keycode.112 = 0x149 # Prior
xkeymap.keycode.117 = 0x151 # Next
xkeymap.keycode.78 = 0x46 # Scroll_Lock
xkeymap.keycode.127 = 0x100 # Pause
xkeymap.keycode.133 = 0x15b # Meta_L
xkeymap.keycode.134 = 0x15c # Meta_R
xkeymap.keycode.135 = 0x15d # Menu

Save it and be happy!

Thursday, June 4, 2009

The FAT16, FAT32 and NTFS file size limitation


The file size limitation appears to be the problem,
but the maximum size depends on the operating system.
What is the size of the file you're copying and where does it reside?

The maximum file size on a FAT32 formatted partition is
4 GB or 4,294,967,296 bytes minus 2 bytes ...

The maximum file size on a FAT16 partition
under Windows 9.x and ME is 2 GB minus 1 byte ...

But the size limit using the FAT16 file system
under Windows NT or 2K is 4 GB ...

And the NTFS file system has a (theoretical) maximum file size of
16 exabytes or 18,446,744,073,709,551,616 bytes (2^64 sectors) ...