Skip to content
April 12, 2013 / jpeg729

PXE booting LinuxMint 14 from SystemRescueCD

I was working on a friends computer whose Windows install had pretty much fried, but I couldn’t get the computer to boot from my external usb cd drive, nor from any usb key, and I didn’t have any cds to hand. So I thought of booting it over the network.

Setup

I ran SystemRescueCD on my laptop, and connected to the wifi network. Then I followed the instructions on their website for PXE booting. It pretty much worked fine.

/etc/init.d/pxebootsrv restart

Normally you should turn off the networks DHCP server because your ordinary router won’t tell the computer where to get the boot files from. I didn’t think of it straight away, but I was lucky. The DHCP server on my laptop responded faster than the router.

Bugfix

(I was running SystemRescueCD 3.4.something, I think)
One config file had an error preventing things from booting. I had to edit /etc/conf.d/pxebootsrv.

nano /etc/conf.d/pxebootsrv

There was a line saying PXEBOOTSRV_LOCALIP="192.168.1.5" and the IP address didn’t match the real IP address of my laptop.

Modifying things for LinuxMint

I had an iso file of LinuxMint lying around.

Mount the iso and copy over the kernel and initial ramdisk

mkdir /mnt/isos
mount drive_with_isos /mnt/isos
cd /tftpboot
mkdir -p images/linuxmint
mount -o loop /mnt/isos/path_to_iso/linuxmint-14.1-cinnamon-dvd-64bit.iso /tftpboot/images/linuxmint
mkdir images/lmin
/bin/cp /tftpboot/images/linuxmint/casper/vmlinuz /tftpboot/images/lmin/
/bin/cp /tftpboot/images/linuxmint/casper/initrd.lz /tftpboot/images/lmin/

Make sure the files are available via nfs

Edit /etc/exports and add “/tftpboot/images/linuxmint *” without quotes in a line by itself.

nano /etc/exports

Then restart nfs.

/etc/init.d/nfs restart

Add an option to the pxe boot menu

Edit pxelinux.cfg/default

nano pxelinux.cfg/default

and add the following in a sensible place

LABEL LinuxMint Livecd 12
KERNEL images/lmin/vmlinuz
APPEND root=/dev/nfs boot=casper netboot=nfs nfsroot=10.10.1.106:/tftpboot/images/linuxmint initrd=images/lmin/initrd.lz nosplash --

Go for it

Worked perfectly and installed just fine. The installation would have run faster if I had plugged a usb key containing LinuxMint directly into the computer I wanted to boot. PXE would just have been a kickstart.

Advertisements
April 8, 2013 / jpeg729

Lightning fast video splitting script

I had a few TV recordings that I wanted to clean up. You know, cut the commercial breaks and trim the extra on the ends. Here is what I did.

Transcode the video files to reduce their size etc.

I used handbrake for that, I like the easy control it gives over things like image cropping, decomb filters and stuff.

MPlayer edl mode

mplayer has a mode in which you can use keyboard shortcuts while watching a video in order to indicate parts to skip. You use it like this.

mplayer video.file -edlout file.edl

You then press i to start a skip section and i again to finish it. There is no visual feedback except in the terminal you used to launch mplayer.

Careful, nothing is written out for a skip that you don’t finish!

The problem

You should be able to use mplayer (and therefore mencoder) to automatically skip those sections like this.

mplayer video.file -edl file.edl

However my version of mplayer (on Ubuntu 12.10) says

Old EDL functionality using the –edl option is not supported.
MPlayer2 UNKNOWN (C) 2000-2012 MPlayer Team

ffmpeg

I had already looked into using ffmpeg for that sort of work. Basically, you can tell it to skip a lot and take just a certain length. And you can tell it to copy over the video and the audio, and it will go through an hours video in seconds flat. However it isn’t always very accurate about the timings because it can only start extracting from some sorts of keyframes in the video.

It is best if you write the results out to a .ts file because ffmpeg can join them up again without hassle.

ffmpeg -acodec copy -vcodec copy -ss $start -t $length -i input.m4v part1.ts

It complained about my input files saying I had to add -vbsf h264_mp4toannexb to make it work. All my input files are in the same format, so I just wrote it into my script. Works for me.

And to join them up again:

ffmpeg -i "concat:part1.ts|part2.ts|part3.ts" -acodec copy -vcodec copy output.mp4

Using my script

I had dreamed about cooking up a nice script, so I did. I called it edl, but you can call it what you like.

I run it like this:
edl videofile.m4v
It doesn’t much matter what format your videos are in as long as mplayer and ffmpeg can read them.
The output will be called videofile.m4v.mp4. I couldn’t be bothered to figure out how to remove the first file extension. I will be renaming the files by hand afterwards anyway.

First it fires up mplayer with the -edlout option, and I mark my skips using the i key. I tend to skip from the very beginning (starting it within the first second will do) up to the beginning of the bit I want. Then I mark the commercial breaks, and finally I mark from the end of the part I want to a bit furthur on (it doesn’t matter how far furthur on).

The edl format thinks in terms of ranges to skip, but my script thinks more in terms of ranges to keep, so if your first skip starts after 10m, my script will give you the first ten minutes of the video. However it wont give you anything after the beginning of your last skip (I got lazy there).

When you quit mplayer, it shows you the contents of the edl file and asks if you want to continue. If you do want to continue, just press Enter, anything else, even “y”, will cause it to exit straight away. (another moment of laziness)

Then it does the ffmpeg magic and a few seconds later it shows you the rough timestamps of the points the resulting video was joined at so that you can check. And finally it fires up vlc so you can take a look.

Advanced use

You can choose to give it an edl file, which is useful for fine tuning and stuff.

You do that like this:

edl video.file edl.file

It will skip the mplayer step and use the provided edl file instead. Careful though, a bad edl file will cause chaos!

Finally, the script

#!/bin/sh

# Colour for text messages
txtrst='33[0m'  # Color off
txtred='33[1;31m' # Red
txtgrn='33[1;32m' # Green
txtylw='33[1;33m' # Yellow
txtblu='33[1;34m' # Blue

toHHMMSS () {
  # Translates a time in seconds to the format required by ffmpeg. HH:MM:SS.mss
  echo `dc -e "$1 60~r 60~r [0n]SadZ2>an [:]n [0n]SadZ2>an [:]n 1~r [0n]SadZ2>an [n]Sad0!=a" | cut -b -12`
}

filename=`basename "$1"`
infile=$1
outfile="$1.mp4"
if [ "x$2" = "x" ]; then
  edlfile=`tempfile`
  mplayer "$infile" -edlout "$edlfile"
  mplayer=true
  echo "Here is what is in the edl file produced by mplayer."
else
  edlfile="$2"
  mplayer=false
  echo "Here is what is in the edl file you specified."
fi
cat "$edlfile"
echo "Continuer? [Y/n]"
read cont
if [ "x$cont" = "x" ]; then

start=0
i=0
joincmd='concat:'
curlength="0"
joins='Joins at'

# Reads in from $edlfile, see end of loop.
while read line; do
  if [ "x$line" = "x" ]; then
    break # Jump out on empty line
  fi

  i=$(($i+1))
  end=`echo $line | cut -d" " -f1`
  startnext=`echo $line | cut -d" " -f2`

  if [ `dc -e "$end 1/n"` -ne 0 ]; then # if wanted an integer, so let's give it one.
    length=`dc -e "$end $start -n"`
    curlength=`dc -e "$curlength $length + n"`
    start=`toHHMMSS $start`
    length=`toHHMMSS $length`

    echo "\n$txtylw Taking part $i from $start length $length\n$txtrst"
    ffmpeg -acodec copy -vcodec copy -vbsf h264_mp4toannexb -ss $start -t $length -i "$infile" "part$i.ts"

    joins="$joins `toHHMMSS $curlength | cut -b -9`"
    if [ $i -eq 1 ]; then
      joincmd="${joincmd}part$i.ts"
    else
      joincmd="${joincmd}|part$i.ts"
    fi
  else
    # First skip starts at 0, so don't take out a part from 0 to 0.
    i=0
  fi
  start=$startnext
done < "$edlfile"

echo "\n$txtylw Joining up the parts\n$txtrst"
ffmpeg -i "$joincmd" -acodec copy -vcodec copy "${outfile}"
rm part*.ts
echo "\n$txtylw $joins"
vlc "$outfile" 2>&1 >/dev/null

if $mplayer; then
  rm $edlfile
fi

fi
August 31, 2012 / jpeg729

Faust tips

Faust is a great framework for programming audio transformation programs and plugins. You write simple code and it compiles it first to C++ then to nearly any sort of plugin/standalone program. You can even upload it to be compiled on their site.

The code you write corresponds to a block diagram that illustrates the pathway the input will take. I found it really intuitive and within a day I had written an adaptive high-pass/low-pass filter with a window full of knobs and sliders to adjust its behaviour.

Here I shall write about a few of the features that took me time to figure out.

How to build a counter

counter = 1 : + ~ _ ;

1 starts a stream of data that just says 1 at each moment in time.

+ takes it in and adds it to its other input

~ is the recursion operator. It takes the output of the + operation and feeds it via the _ operator back in to the +

_ just lets its input pass through unchanged, that is why it doesn’t have a block for itself in the diagram

; tells the compiler the line has finished

So at time 0, the plus receives 1 and outputs 1

At time 1, the plus receives 1 and 1 from the delayed recursion, and outputs 2

At time 2, the plus receives 1 and 2 from the delayed recursion, and outputs 3

And so on.

Counting seconds

import("math.lib")
seconds = (1 : + ~ _ ) / SR

The import is needed in order to use SR = the sample rate.

The counter above goes up one for every sample. Divide by the sample rate and you have the number of seconds as a float. If you want a whole number use a cast. Like this…

seconds = (1 : + ~ _ ) / SR : int

Understanding smoothing

In filter.lib there is a useful function for lazily following the amplitude of a signal. It is called smooth, it takes one parameter to tell it how smooth to be. Combine it with tau2pole to have a better idea how fast it smooths. For example

smooth(tau2pole(number_of_seconds)) will get to within about 0.63 of a constant signal in number_of_seconds, and will get very close in 4 times that amount of time.

Here is the block diagram of smooth(0.9)

If you look closely you can see a line coming in from the left, it represents the input signal. The arrow going out on the right is a smoothed version.

Here is a breakdown of how it works.

At time 0, we multiply the input signal by 1 - smoothing_constant. Here that will be multiplying by 0.1. That value will pass though the + unchanged. Were are not very close yet.

At time 1, the recursion loop has picked up the previous output and multiplies it by the smoothing constant. Here that will be 0.9. We add to that 0.1 of the original signal as before, and we have 0.1 * signal + 0.1 * 0.9 * signal. That is closer to the original signal, and the estimate is just going to get better.

The smoothing constant can be chosen between 0 and 1, the smaller it is the faster it gets there, and the larger it is the slower it gets there. If you want to know how long it is going to take use tau2pole to calculate the smoothing constant, otherwise the smoothing speed will change depending on the sample rate of the input.

Remember: we are calculating all that based on a given input signal, which could be changing all the time. So we will be chasing it down all the time.

Graphical user interfaces

To start with read the tutorial pages 18 to 20. Now you know all about buttons, sliders and so on. Read this too. Even after that I hadn’t understood how to group and order the controls in my window. Then I studied some example code.

Grouping

I thought that to group controls there had to be a line somewhere saying hgroup("name", list of controls), but each control returns a number that will be used in some calculation or another. Didn’t seem logical.

Here is how it works.
Each time you run hgroup("name", hslider(...)) it adds a slider to the group called “name”. So instead of writing *(hslider("Gain", ...)) you can write *(hgroup("name", hslider("Gain", ...))) and it will be in the right place.

Now for some shorthand. Suppose I define a function like this
volume_controls(x) = hgroup("Volume controls", x);
Then I can just write *(volume_controls(hslider("Gain", ...))) and I know that my Gain slider will appear in the right group in my programs window.

You can even nest several levels of these to put groups within groups, groups next to simple controls, etc…

highpass_group(x) = vgroup("Highpass filter", x);
highpass_contents(x) = highpass_group(vgroup("Parameters", x));

Then you can put an enabled button in the highpass_group and then the controls for the parameters in a subgroup. Neater that way.

Ordering

In each group the controls appear in alphabetical order of their names, so it seems we are stuck with that. However if you put a word in square brackets in the name it disappears, so we can just start the names with a number in square brackets, e.g. "[1] Gain" or "[2] Treble" or "[3] Base".

Don’t put a colon in there because it might be treated like a parameter, for example, [style:knob] in the name will turn a slider into a knob if you compile in some formats. See the end of this page for more parameters.

Conclusion

Hope that helps you have more fun faster.

July 19, 2012 / jpeg729

Lenovo x200 tuning

Here is what I did to tune up my lovely Lenovo x200.

UPDATE for Ubuntu 13.04. The fan seems pretty quiet anyway and I have already done some compiling.
The modification of the linux command line is still needed.

Install some thinkpad stuff
sudo apt-get install tp-smapi-dkms thinkfan

tp-smapi-dkms enables enhanced battery stats and charging control.
thinkfan allows software fan control. Mine would spend hours pumping out cold air.

To make sure relevant modules will be loaded add the following lines to /etc/modules

tp_smapi
thinkpad_acpi

Hard drive shock protection

I just installed hdapsd and it works.

Get it to actually turn off when you shutdown

Edit /etc/default/grub changing the line that reads
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"
to read
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash acpi_osi=Linux"
That also fixes the mute button.

Better battery charge control

To make the battery last longer avoid recharging it if it is only a little bit discharged, etc. as described here. Another tip: Never let the battery fully discharge, that hurts lithium ion batteries each time.

# echo 50 > /sys/devices/platform/smapi/BAT0/start_charge_thresh
# echo 90 > /sys/devices/platform/smapi/BAT0/stop_charge_thresh

You probably have to add those lines to /etc/rc.local to make them stick.

Fan control

Firstly you have to enable fan control, by creating /etc/modprobe.d/thinkpad_acpi.conf with the following contents.
options thinkpad_acpi fan_control=1

Then enable thinkfan by changing START=no to START=yes in /etc/defaults/thinkfan
You can edit /etc/thinkfan.conf to tune the fan behaviour if you like.

CPU undervolting

See here.

July 10, 2012 / jpeg729

Kernel compiling to add phc support – lowlatency kernel

DISCLAIMER: I am *not* an expert. I just read docs and try to figure out something that works, and I really don’t know how or why it works.

Get the sources and everything else needed, I’m doing this for the lowlatency kernel that comes with UbuntuStudio 12.04.

apt-get source linux-image-3.2.0-23-lowlatency
apt-get build-dep linux-image-3.2.0-23-lowlatency
apt-get install libncurses5-dev

Do that in the directory you want the kernel source to land up in.

Move into the source directory

cd linux-lowlatency-3.2.0

Copy over the current lowlatency config into a new config.

cp /boot/config-3.2.0-23-lowlatency debian.lowlatency/config/amd64/config.flavour.lowlatency-phc

In the following the debian/rules commands ought to be run using fakeroot, e.g. fakeroot debian/rules updateconfigs but I am lazy and I am doing everything as root anyway.

debian/rules updateconfigs

It got stuck twice saying there was a permissions problem on some script. I did the following each time and re-ran it.

chmod +x the-script-path-and-name

Now we edit the config.

debian/rules editconfigs

It will ask you whether you want to modify various configuration files. Given that I have an intel Core2Duo P8400 I said I’d modify the amd64/lowlatency-phc configuration. The options that need to be changed are the following. These things need to be compiled as modules rather than included in the kernel. That way the phc module can replace them.

Power management and ACPI options --->
  CPU Frequency scaling --->
    x86 CPU frequency scaling drivers --->
      <M> ACPI Processor P-States driver
      <M> AMD Opteron/Athlon64 PowerNow!
      <M> Intel Enhanced SpeedStep (deprecated)

Given that we have created a new config there are a few files to create/edit so that it will work.

First edit debian.lowlatency/etc/getabis adding lowlatency-phc to the line that reads getall amd64 lowlatency. Given that we will use skipabis later on, this might not be necessary.

Next edit debian.lowlatency/rules.d/amd64.mk adding lowlatency-phc to the line that reads

flavours = lowlatency

Create a description file for this flavour. You can modify it if you like.

cp debian.lowlatency/control.d/vars.lowlatency debian.master/control.d/vars.lowlatency-phc

Had to edit debian/control aussi.
Found these lines and modified the first to read linux-image-3.2.0-23-lowlatency-phc

Package: linux-image-3.2.0-23-lowlatency
Architecture: i386 amd64
Section: universe/admin

Likewise here to read linux-headers-3.2.0-23-lowlatency-phc

Package: linux-headers-3.2.0-23-lowlatency
Architecture: i386 amd64
Section: universe/devel

Cheating really ‘coz now the plain lowlatency flavour won’t build. I ought to have copied the entire sections.

Finally let’s compile it.

skipabi=true
debian/rules binary-lowlatency-phc

Last but not least, install it. Don’t use sudo if you are already root.

cd ..
sudo dpkg -i linux-headers-3.2.0-23-lowlatency-phc_3.2.0-23.36~precise1_amd64.deb
sudo dpkg -i linux-image-3.2.0-23-lowlatency-phc_3.2.0-23.36~precise1_amd64.deb

The phc-intel module was automatically built and installed by the dkms system. Nice.

Rebooted and it works!!!

Now I have added the following lines to /etc/rc.local to make the undervolting changes stick. You will need to experiment as described here to find the values right for your CPU.

# Undervolt the cpu cores.
echo '22 19 18 17' > /sys/devices/system/cpu/cpu0/cpufreq/phc_vids
echo '22 19 18 17' > /sys/devices/system/cpu/cpu1/cpufreq/phc_vids

Adapted from http://blog.avirtualhome.com/compile-linux-kernel-3-2-for-ubuntu-11-10/

July 9, 2012 / jpeg729

Undervolting an intel P8400 CPU in a lenovo x200 thinkpad

First I followed these instructions, with a few differences.

I didn’t use the testing ppa. I did have to install msr-tools to get the analysis page of phctools working.

Then I followed these instructions, with a few differences.

Firstly some useful little scripts.

Here is my script to set the cpu frequency

#!/bin/sh
if [ -n "${1}" ]; then
    sudo echo "Setting at ${1}MHz"
    for i in 0 1; do
        sudo cpufreq-set -c $i -g userspace
        sudo cpufreq-set -c $i -f ${1}MHz;
    done
fi
sudo cat /sys/devices/system/cpu/cpu*/cpufreq/cpuinfo_cur_freq
Run it like this: setcpufreq 800
or without an arg to see what the current frequencies are.

Here is my script to stress the system.

#!/bin/sh
burnP6 &
burnP6 &
burnMMX &
burnMMX &

function ctrl_c() {
    killall burnP6
    killall burnMMX
}
trap ctrl_c INT

while sleep 5s; do
    date > still_running
    cat still_running
done

While it is stressing out the system it writes the time & date to the file still_running every 5 secs and when you do Ctrl-c it kills the CPU hogs.

Here is my script to set the vids.

#!/bin/sh
sudo echo "Setting vids to ${1} ${2} ${3} ${4}"
vids="${1} ${2} ${3} ${4}"
for i in 0 1; do
    cmd="echo '${vids}' > /sys/devices/system/cpu/cpu${i}/cpufreq/phc_vids"
    sudo sh -c "${cmd}"
done
cat /sys/devices/system/cpu/cpu*/cpufreq/phc_vids
done
Use: setvids 43 35 23 17
It then shows you the values that have been stored, which aren’t necessarily taken into account by the CPU, see the analysis page of phctools to see what is really happening.

My simpler test procedure

My vids were 43 35 23 17 to begin with.
The vids for 800MHz are already as low as they can be, so we won’t worry about those.
The system seems to be only able to put one cpu in 2267MHz and then only sometimes and just for a mo or two. So we won’t worry about that one yet.
So I put the system in 2266MHz, started stressing the system and lowered the vids for 2266MHz every 15-30 secs until the system starts going pear-shaped. For me it just showed a few strange characters on a black screen and then showed me the login screen again. That was at 17.
Next I upped the vids by 2 steps and tested for a good while at 19. Then I put the cpu at 1600MHz and the vids for 1600 at 18, and tested for a good while.
Final result, I decided to put them at 22 19 18 17. I fixed on 22 for 2267MHz to give it some margin. I don’t really know what is going on for 2267MHz nor how to test it properly. I think it might be the strange turbo mode that exists in some Intel CPUs.
Finally to make things permanent I added the following to /etc/rc.local
# Undervolt the cpu cores.
echo '22 19 18 17' > /sys/devices/system/cpu/cpu0/cpufreq/phc_vids
echo '22 19 18 17' > /sys/devices/system/cpu/cpu1/cpufreq/phc_vids

Next challenge – make it work on a realtime kernel