Twisted IT paired

Allright.
I've got a server (actually my desktop testing machine) with two brand new installed 2x1T disks.
I'm going to setup the disks like this:
3 partitions:
1 swap (we really don't need abstractions for just keeping swap)
2 boot partition in md raid1 (grub2 really sux, so no boot from lvm support in the old one...)
3 all othe space for md raid1 and lvm over it.
It's a good idea to use lvm because you can always add another disk and also can make snapshots... and in short - have more fun with space allocating.


1. partition the two disks identically:
#> fdisk -l /dev/sdb
Device Boot Start End Blocks Id System
/dev/sdb1 1 974 7823623+ 82 Linux swap / Solaris
/dev/sdb2 975 1461 3911827+ fd Linux raid autodetect
/dev/sdb3 1462 121601 965024550 fd Linux raid autodetect

(yes, I know the boot partition is quite big, but there is a lot of space and I prefer to have more space than to wonder wtf I've done... happened few timesof course )


2. create raids.
#> mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb2 /dev/sdc2
and activate
#> mdadm --readwrite /dev/md0
be sure it's sync-ing
#> cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdc2[1] sdb2[0]
3911744 blocks [2/2] [UU]
[=>...................] resync = 5.2% (206848/3911744) finish=0.5min speed=103424K/sec

do the same for lvm raid partitions....


3. format boot partition (I'll use etx3) and copy boot files in there.
#> mkfs.ext3 /dev/md0
#> mount /dev/md0 /mnt/
#> cd /mnt/
#> cp -a /boot/ .
#> cd grub
WARNING: sda2 is supposed to be your current partition.
#> sed -i'' -e 's/\/boot\//\//g' -e 's/sda2/mapper\/1tb-root/g' menu.lst
#> cd /;unmount /mnt


4. create physical volume on md1.
#> pvcreate /dev/md1
Physical volume "/dev/md1" successfully created


5. create volume groups - my vg name 1tb
#> vgcreate -A y 1tb /dev/md1
Volume group "1tb" successfully created


6. then add root volume group. mkfx.ext3.mount and copy the currently running root system in new root partition...
#> lvcreate -A y -L 30G -Z y -n root 1tb
#> mkfs.ext3 /dev/1tb/root
#> mount /dev/1tb/root /mnt
#> cd /mnt
#> cp -a {/bin,/cdrom,/emul,/etc,/home,/initrd*,/lib,/lib32,/lib64,/media,/opt,/root,/sbin,/selinux,/srv,/tmp,/usr,/var,/vmlinuz*} .
#> mkdir dev proc sys mnt misc boot
#> cd etc
#> sed -i'' -e 's/sda2/mapper\/1tb-root/g' fstab
WARNING: There is a nasty bug with initramfs tools described here: http://www.mail-archive.com/debian-kernel@lists.debian.org/msg32272.html
You MUST set root to /dev/mapper/VGNAME-LVNAME otherwise you won't get lvm support in your kernel.
#> echo "/dev/md0 /boot ext3 defaults 0 0" >> fstab
#> mount -o bind /dev /mnt/dev
#> mount -o bind /prov /mnt/proc
#> cp -a /boot/ /mnt/boot/
#> chroot /proc
#> update-initramfs -u -t -k `uname -r`
#> exit
reboot the machine, edit the grub menu by hand to boot from (hd0,0) as boot and load /dev/mapper/1tb-root as root.
login as root and make:
#> cd /boot; mkdir oldrootfs; mv oldrootfs; mv -fr oldrootfs/boot/* .;
edit grub/menu.lst to have / instead ot /boot/ dirs.
run:
#> grub-install --root-directory=/ "(hd0)" (described in /usr/share/doc/grub/README.Debian.gz
(root-directory is where the boor partition is on hd0, if you put /boot/ then in live system you'll have /boot(md0)/boot/)...
reboot again and it must be ok now.

7. Install grub on both disks as described here...

8. reboot and disable hdd from bios. the system should boot normally and you should see that you are using the lvm root partition.

I'm migrating a xen machine (my gallery - g.pechurka.com ) from one machine to another.
I wanted to research from quite a long time what should I use in Linux.
I've got a nice working setup but when you research and test you always makes things better.

So, this night I've researched on if I should do MD raid1 or LVM2 mirroring.
Short answer: MD0 raid1 and Physical Volume over it (as used to now). Sometimes LVM is faster but when using single file. You can have serious problems if you have power failure or disk failure/disconnection when using LVM mirroring. It relies on underneath layer to write caches. For further reading open the link below.
Long answer here: lvm2-mirrors-vs-md-raid-1

I've found PureEdit today and decided to use it as my CMS backend for some of my simpler projects!
This is a GREAT and FAST way to setup a backend content editing.
Take a look at the videos on their site! I was fascinated!

I've had old database filled with correct Unicode (UTF-8) data (in Cyrillic).
I've installed and loaded pe-admin, updated my db accortind to PureEdit specs, added some fields type like (status 1||0).
When opened pe-admin I've come up with few Unicode problems.
(as described here: http://www.pureedit.com/community/comments.php?DiscussionID=149&page=1 )
I've solved my problem exactly by putting: mysql_query('SET NAMES UTF8');

I've had to modify the connect function in pe-admin/databases/mysql.db.php like this:
function connect($host, $username, $password, $database)
{
$dbh = mysql_connect($host, $username, $password);
mysql_query('SET NAMES UTF8');
mysql_select_db($database, $dbh);
}
(it used to connect/select db in one line, but you MUST make SET NAMES BEFORE selecting db!)

The second problem that I've had as with utf8 again.
Utils class is not UTF-8 ready.
I've had to add to set utf8 encoding
public function __construct() {
mb_internal_encoding('UTF-8');
}
and replace all str functions with mb_str.

After that I wanted to have pagination.
Here it's described how to set it up.

http://www.pureedit.com/community/comments.php?DiscussionID=221&page=1#Item_0

Now I have to setup host type filed (that will be automatically filled with $_SERVER['REMOTE_ADDR']) and some othe fancy stuff, but in about 40 minutes I've setup a GREAT and easy to use backend posting solution for my website!

Thanks PureEdit

The PureEdit code is not from the greatest ones but when it works and if .htaccess protected - it simply works!

How do we understand when a project is well coded or is a piece of..... fish?
Well, have you heard of WTFs per minute?
See this

thanks to OSnews.com/comics for this one!

Just a simple test for speed of ext2, ext3, ext4dev, reiserfs and xfs.
I've created a 5G lvm and did mkfs.* to it with all these systems.
lvm> lvcreate -L 5G -n speedtest tvg
mkfs (all default opts)
mount (no opts)
Then make dd if=/dev/zero of=a bs=G count=3
Then make dd if=a of=b


Here are the results:

FS type	dd if=/dev/zero of=a speed	dd if=a of=b speed
ext2	3221225472 bytes (3.2 GB) copied, 225.283 s, 14.3 MB/s	2047094784 bytes (2.0 GB) copied, 38.4914 s, 53.2 MB/s
ext3	3221225472 bytes (3.2 GB) copied, 248.433 s, 13.0 MB/s	1912479744 bytes (1.9 GB) copied, 52.1087 s, 36.7 MB/s
ext4	3221225472 bytes (3.2 GB) copied, 281.699 s, 11.4 MB/s	1918500864 bytes (1.9 GB) copied, 49.1343 s, 39.0 MB/s
reiserfs	3221225472 bytes (3.2 GB) copied, 248.827 s, 12.9 MB/s	2108379136 bytes (2.1 GB) copied, 62.5061 s, 33.7 MB/s
xfs	3221225472 bytes (3.2 GB) copied, 426.823 s, 7.5 MB/s	2132619264 bytes (2.1 GB) copied, 55.3356 s, 38.5 MB/s

It seems like ext3 is working best with big continious files currently.
Of course this is not a real life situation.
Maybe xfs and reiser will be better for a lot of small files? (reiser has it's glory about that).

Please let me know what you think!

Have you ever been in the middle of deployment and noticed that the owner and permissions on dev package you have and production one are different?
Very oftern when deploing some sites I have two identical dirs with different owners and permissions - one from a dev web server and another from the production (where everything is tuned and working).
Here is a quick and durty way to change all ownership and permissions from prodution to development dir.
BE VERY CAREFULL. These permissions are not all masks. These are the one I needed!
Don't blind apply this script and run chm.sh after that!!!
FIRST check that you don't have some permissions rules in the chmod!!!
If you have 'chmod rw-r---- FILENAME' you will finish with -------- (0000) permissions!!!
You were warned!
mysite.production is a copy of the production dir somewhere else in the tree (your home for example).


$> find mysite.production/ ! -type l -ls | awk '{print "chmod "$3" "$11 " && chown "$5":"$6" "$11}' | sed -e 's/\.production//g' -e 's/-r--r--r--/044/' -e 's/-rw-rw-rw/0666/' -e 's/-rwxrwxrwx/0777/' -e 's/-rwx------/0700/' -e 's/-rwxr-xr-x/0755/' -e 's/-rwxr-x---/0750/' -e 's/-rw-r--r--/0644/' -e 's/-r-x------/0500/' -e 's/drwxr-xr-x/755/' -e 's/drwxrwxrwx/777/' > chm.sh
$>sh chm.sh


and try to compare some files that have specific perms.

that's all.
I'm SURE there is an easier way, and I'll be glad to share!!!

Short story:
If you have a machine that you want to have XEN+3D use ATI with radeonhd driver (check wich cards are supported).
Take a look here before try more than basic configuration of the driver: http://wiki.debian.org/XStrikeForce/HowToRandR12.

Long story:
For more than 5 months I've had a machine at home that was always off.
But not until tonight.
It is a Core 2 Quad @ 2.5 GHZ with 4 gigs ram - a killer for desktop, very descent for server.
You'll almost never need that kind of power (expect you are a mad scientist doing some kind of calculations or a crazy gamer - not me!).
You'll definitely don't need it if you only watch movies, play some stupid doom like games, browse, read pdfs and do all kind of daily stuff on it.
When I got it I told to myself that I'll never use it until I can both have it as my test deployment server and as my primary desktop (including a dual headed video with two monitors connected and using two different GNOMEs on each).

You'll tell to yourself: 'uh! easy stuff.' (at least that's what I thought)
Well here I am five months later, at last I was able to complete my setup.

I've started by installing Debian Lenny (current unstable - now stable) and started digging....
The easiest part was configuring the dual head card to use the two separate monitors and to have two totally separated GNOMEs running.
It took me about two days to figure out that. It turned out that it is impossible to do this with only two X servers.
That is because they can't get attached to the same hardware card (if you have two different hardware cards - no problems).
The trick was to start a X server with wide screen splited on the two monitors (so called Xinerama or dual-view-mode - you know when you can move a window from one monitor to another). Upon this server you get another two 'virtual servers' started each running a GDM.
I'll find the link describing this and put it here.

So far, so good - I've got the two separate X working perfectly and I was almost complete.
Last step was just to change the kernel with XEN enabled one and I'm done.
When I rebooted in the XEN kernel I was 'WOW---F**KING' ... the XEN kernel was not able to work with Nvidia proprietary drivers.
At that time the machine was with NVIDIA GForce 7600 card.
I dug few weeks just to find nothing new - there were two ways to use NVIDIA (drivers):

• stop using XEN


• patch your kernel and hope you get it running and hopefully you'll have no problems

I didn't wanted to change my vanilla debian kernel because this is going to be my test deployment server and it should be as close as it can to my production machines.
Until then NVIDIA is dropped from my favorite video cards list - sad but true, it was my leader and most recommended card to be used it with linux...
What can you think of after NVIDIA?
ATI of course!
I have one - Asus X1600 (ATI chipset: RV530 X1600).
I've plugged it in. At that time I knew that only proprietary drivers were available for newer ATI cards, otherwise you get VERY bad performance and no 3d. That's why I've installed fglrx driver simply to find out that they also can't work with XEN.
DAMN.
The situation seemed hopeless. What other card can you use? (matrox? they've reports to have troubles with xen too!) and have descent 3d in linux? (please If you are aware of any card, post into the comments!)
I've left the project for few months but from few days while talking with a friend of mine, I've started looking for alternative of NVIDIA and ATI.

Tonight I was very surprised when I saw someone mentioned radeonhd driver. I've never heard of it. I've started digging and in about an hour I've got my ATI running with radeonhd driver and showing about 1500 FPS in glxgearx (WOW!!!) with XEN ENABLED KERNEL!
What was my suprise to see this. My problem was solved!
Just lost another hour to figure out how to start my two monitors split ed out in two with one on the left of the other... Xinerama won't work with this driver.
To fix things as you want use xrandr command ( from irc.freenode.org #radeonhd hint)
You can read more here: http://wiki.debian.org/XStrikeForce/HowToRandR12
Be careful with the device names.
These commands did the job for me:

xrandr --output DVI-I_1/digital --off
xrandr --output DVI-I_1/digital --mode 1680x1050 --rate 75
xrandr --output VGA_1 --left-of DVI-I_1/digital

Now I'm happy opensource driver user with most debian vanilla system I can imagine.

As a programmer I have a nice cool helpers - classes I've written to help me do my daily work.
Such class is my Mailer class which I use almost daily.
It has all kind of fancy options and can do all kind of stuff - from sending simple mail to just setting an array with files which should be attached in the mail.
It used to work very good until today.
I was very surprised when I saw a broken email coming from it.
I've investigated a few minutes what has changed and I've found that the problem
I've set
mbstring.func_overload = 7
according to the description in file:
; For example, 7 for overload everything.
; 0: No overload
; 1: Overload mail() function
; 2: Overload str*() functions
; 4: Overload ereg*() functions
it turned out that when set to 7 instead of using mail() function, php starts using mb_send_mail.
mb_send_mail works only with base64 emails.
It automatically encodes the body and subject (but not the sender wtf?) and it breaks mail ugly!
If you manually add/set headers do not put \r\n at the very end of the string, mb_send_mail adds two headers after yours:
Mime-Version: 1.0
Content-Transfer-Encoding: BASE64
and if you put one new line(\r\n) to separate the headers from body then you get the email totally messed up, because the base64 transfer won't apply and you'll get encoded string as plain text or html or whatever content-type you set previously.
After I've lost another 20 minutes figuring out this, I've finally achieved receiving my email as it should be... well... almost.
When the mail contained only ASCII symbols everything was ok, but when I tried to send email in utf8 Bulgarian... omfg?!
The encoding was broke and for some reason the UTF8 symbols looked like you have page in UTF8 but your browser is set to WINDOWS-1251....
WTF?!!! I've read few posts that said they never got it successfully working fine with mb_send_mail.
It took me some time to find something very useful - there is an undocumented php function: mb_orig_mail (http://osdir.com/ml/php.internationalization/2003-01/msg00054.html).
When you use overloaded function you can fall back to the standart mail function by calling this one.
So here is what I did to solve my problem - I use exactly the same class that used to work until the overload get active.
But when it comes to sending I simply check if the mail function is overloaded and if yes, then call the mb_orig_mail.
Here is the actual code:

$mailFunct = 'mail';
if (ini_get('mbstring.func_overload') == 1 || ini_get('mbstring.func_overload') > 4) {
$mailFunct = 'mb_orig_mail';
}
if(! $mailFunct($tostr, $subject, $tmpBody, $tmpHeaders) ) {
echo 'sent';
} else {
echo 'error sending';
}

It's as simple as that.
Hope that helps to anyone.
Comments are welcome.

In my daily work, I need to test sites I work on with a lot of browsers.

Another common problem is testing a website as a different users - anon users, registered users, admins etc.

This is the list of browsers I use in Linux and Windows.

in my Linux (Debian stable or Ubuntu depends on which machine):

1. Firefox (www.getfirefox.com) - my favorite! I use it on hourly basis . It's a lot of features, plugins and of course something that's a must for a web dev - firebug.


2. Opera (www.opera.com) - my second choice - Awesome browser!!! Feature rich, perfect for casual user. It has integrated mail client, torrent client. Also has a developer toolbar, but It was hard for me to get used with.


3. Galeon (galeon.sourceforge.net)- advanced gnome web browser. uses gecko engine. pretty thin and quick.


4. Epiphany (projects.gnome.org/epiphany) - the default gnome web browser. Light, solid, gecko based, won't eat up your memory as firefox does. Of course it misses some features.


5. Konqueror (www.konqueror.org) - Default KDE3 web (and file) browser (as all KDE apps - qt based). If you have KDE (and all qt as dependent) libs installed, you can use/install it, otherwise - no point installing so much bloat for nothing. This one uses khtml engine but I've faced a lot of
   annoying problems while using it. It was removed from kde4 [maybe I'm wrong?but it's not default filebrowser anymore? correct me if I'm wrong].

in Windows (XP Pro, yes I have a license coming with my laptop):

1. Firefox
   


2. Opera


3. Chrome - very light. very fast. uses khtml engine for html/css representation. pretty cool, but not for me. I prefer opera if want a light experience.
   


4. Safari - apple browser, but for windows. Uses khtml engine. I don't like it very much, but it's pretty slick (as all mac programs). Used only for testing
   


5. Internet explorer7 (and 6, 8 in different VirtualBox machines. DUMB MicroS!!! They don't provide descent way of installing all three on one windows!!!) - maybe I shall keep silence the link below describes it better.

more funny reading: If browsers were women.

I hope this article is helpfull. Please share your experience with me.

Thanks!