Apache httpd deflector shields

Today I deceided to research which HTTP status codes are supported by the Apache httpd server, browsing around the source code repository [1] I found the http_protocol.c file which describes the implemented http status codes in the daemon.

As part of my good practices I have developed an ErrorDocument template that I call "Deflector Shield" which returns a 302 status instead of 404, 403 or the awful 500. According to the official documentation [2], the directive goes as follows:

ErrorDocument <StatusCode> <Document>
Where
  <StatusCode> is one of the implemented in the source code [1]
  <Document> is an error message or the path to a resource (either local or remote)

All this works for status codes other than 401 (Authorization Required) which require the message to be either the hardcoded or a custom string.

The custom error document directives are here [3]
[1] http://svn.apache.org/repos/asf/httpd/httpd/branches/2.2.x/modules/http/http_protocol.c
[2] http://httpd.apache.org/docs/2.2/mod/core.html#errordocument
[3] https://gist.github.com/4015668/

--
  = ^ . ^ =

Zotero => bibtex with md5 identifiers

I was just hesitated to see annoying useful BibTeX ids like: "???_tonejito_???" so I found this page [1] that shows how to mess with the Zotero Firefox plugin, and I came out with the idea of using the md5 of the title as identifier (for me that is better than the other id).

The file in question was <FirefoxProfile>/zotero/translators/BibTeX.js the whole code is javascript so it's easy to modify and play with.

Looking around the web I also found this github:gist [2] to calculate an md5 sum from a given string, I pasted all the code at the end of the file and used the md5_hex function as follows to replace the element key.


$ diff -u /tmp/BibTeX.js BibTeX.js 
--- /tmp/BibTeX.js 2012-10-31 02:51:49.000000000 -0600
+++ BibTeX.js 2012-10-31 02:51:43.000000000 -0600
@@ -2083,7 +2083,8 @@
  if(!type) type = "misc";
 
  // create a unique citation key
- var citekey = buildCiteKey(item, citekeys);
+ //var citekey = buildCiteKey(item, citekeys);
+ var citekey = hex_md5(buildCiteKey(item, citekeys));
 
 
  // write citation key


I'm currently using the Zotero plugin [3] and the zotero bibtex auto-exporter plugin [4], I tested this using the preferences pane for the autoexporter and run a manual test. After a couple of tries the results were like this:



@misc{20c7358045528f33804340fb6510b8b9,
title = {tonejito {(Andres} Hernandez) · {GitHub}},
url = {https://github.com/tonejito/},
urldate = {2012-10-31},
howpublished = {https://github.com/tonejito/},
file = {tonejito (Andres Hernandez) · GitHub:/Users/tonejito/Library/Application Support/Firefox/Profiles/profile.default/zotero/storage/AAAAAAAA/tonejito.html:text/html}
},

@misc{823e4a8551f18d37794d3115226700ba,
title = {Tonejito},
url = {http://tonejito.blogspot.com/},
urldate = {2012-10-31},
howpublished = {http://tonejito.blogspot.com/},
file = {Tonejito:/Users/tonejito/Library/Application Support/Firefox/Profiles/profile.default/zotero/storage/BBBBBBBB/tonejito.blogspot.com.html:text/html}
},

@misc{d83e6f346506b44510957a2fa00fff13,
title = {Andres Hernandez {(Tonejito)} on Twitter},
url = {https://twitter.com/tonejito},
urldate = {2012-10-31},
howpublished = {https://twitter.com/tonejito},
file = {Andres Hernandez (Tonejito) on Twitter:/Users/tonejito/Library/Application Support/Firefox/Profiles/profile.default/zotero/storage/CCCCCCCC/tonejito.html:text/html}
}



So I can use any of this hashes in a LaTeX document like this and the bibliography entries will be sorted and referenced correctly.

\textbf{Twitter}
\cite{d83e6f346506b44510957a2fa00fff13}


\textbf{Blog}
\cite{823e4a8551f18d37794d3115226700ba}


\textbf{Github}
\cite{20c7358045528f33804340fb6510b8b9}



Thanks LaTeX and BibTeX

[1] http://www.curiousjason.com/zoterotobibtex.html
[2] https://gist.github.com/951664
[3] https://addons.mozilla.org/en-US/firefox/addon/zotero/
[4] https://addons.mozilla.org/en-US/firefox/addon/zotero-bib-autoexport/
--
  = ^ . ^ =

Speeding up SSH logon

A firewall blocks port 53 for my servers so ssh is trying to resolve my address every time I attempt log in.

The timeout is frustrating, so I read the man page [1] and found this configuration directive

  UseDNS  Specifies whether sshd(8) should look up the remote host name and check that the resolved host name for the remote IP address maps back to the very same IP address. The default is ''yes''.

I set it to 'no' and restarted the service and it worked like a charm :D

Note:
  When working on OpenBSD, be sure to be logged in on the console by other method (like physical console access or serial) because restarting the ssh service on OpenBSD causes all remote sessions to be closed.

[1] http://linux.die.net/man/5/sshd_config

--
  = ^ . ^ =

Random Password Generator

N := factor of 3


% N=9 ; (dd if=/dev/urandom bs=$N count=1 | uuencode -m - | sed -n '2p') 2>/dev/null
--
  = ^ . ^ =

% make FF

% cat Makefile


FOLLOWERS?=
ME?=Tonejito
SIGNATURE?="= ^ . ^ ="


FF:
if [ "${FOLLOWERS}" ] ; \
then  \
 for FOLLOWER in ${FOLLOWERS} ;  \
 do  \
   echo "FF @$$FOLLOWER" ;  \
 done ;  \
else  \
 echo "FF @${ME}" ;  \
fi ;
echo ${SIGNATURE} ;
--


% make -s
FF @Tonejito
= ^ . ^ =


% make -s FF FOLLOWERS="alpha beta gamma"
FF @alpha
FF @beta
FF @gamma
= ^ . ^ =


--


crontab -e



# m h  dom mon dow   command
  0 12  *   *   5    make FF

--
= ^ . ^ =

Kill annoying processes that match a pattern

So, there was a bunch of annoying processes named wit a pattern and I wanted to kill all of them

Here is the script (I know this can be done in a much cleaner way in awk, but I like this way)


#!/bin/sh

P="master(-worker)?"

PS=/bin/ps

SED=/bin/sed

KILL=/bin/kill

GREP=/bin/grep

CUT=/usr/bin/cut

$KILL `$PS ax | $GREP -E $P | $GREP -v grep | $SED -e 's/^\ \+//g' | $CUT -d ' ' -f 1 | $GREP -E '^[[:digit:]]+'`

--

  = ^ . ^ =

VirtualBox serial console on Mac OS X

I normally access a VirtualBox VM serial console through minicom in GNU/Linux, but for some reason it didn't worked on Mac OS X, so I researched about how can I access the serial console.

VirtualBox (and VMware for what I saw on the pages [1]) map the serial device of the virtual machine to a "Named Pipe" (actually a UNIX Domain Socket) which can be accessed using netcat or minicom. I tried the nc -U variant as stated in the page but I had no luck making it work because the mac ports version of netcat does *not* support attaching to UNIX domain sockets [2].

The screen man page [3] describe how to attach to a existing tty device but the file is a named pipe so the program cannot do its magic with it. There were some pages describing how to use socat [4] to map a UNIX domain socket to a tty device (actually a PTY) [5] [6] [7]. I also found a couple VMware forum posts with useful links [8] [9].

Thanks to this I can happily run the following two commands to attach to the serial console

% socat -d -d ./Thesis.ttyS0 PTY
2012/05/29 01:08:00 socat[29713] N opening connection to LEN=16 AF=1 "./Thesis.ttyS0"
2012/05/29 01:08:00 socat[29713] N successfully connected from local address LEN=16 AF=1 ""
2012/05/29 01:08:00 socat[29713] N successfully connected via
2012/05/29 01:08:00 socat[29713] N PTY is /dev/ttys007
2012/05/29 01:08:00 socat[29713] N starting data transfer loop with FDs [3,3] and [4,4]
In this case the desired PTY is /dev/tty007 and then in another terminal window

% screen /dev/ttys007

I wanted to do this as painful (automated) as possible, but there were a few problems:

1. The PTY device allocated on Mac OS X is subject to the number of terminals currently being used, so it is a variable device.
2. I have to run two commands in order to get the PTY and attach to it (socat, then screen)
3. Simple shell magic can't work because socat outputs the messages to stderr.

I decided to do some shell magic to automate the task in several steps

1. Map the UNIX domain socket to a PTY
2. Somehow, get the PTY name (the tricky part)
3. Once the name is known, attach to the PTY

Map the socket to a PTY

Same as above, still the messages are output to stderr

socat -d -d ./Thesis.ttyS0 PTY

Get the PTY name

The device name is printed to stderr and simply by doing a 2>&1 and pipe it through a sed or awk instance will do the trick. NO!!!, for some reason (still unknown to me) sed and grep got stuck and even if they got the apropriate input they didn't send anything to the screen.

I managed to solve the problem by redirecting the socat output to a file (which might also be a FIFO if you are interested), and then pointing grep to get the desired line and piping that output to sed to clean the text and get the desired PTY name.

Making it work altogether

Since the device is dynamically allocated and the socat output may vary, this kind of magic can be done, yes it might also be done with an sed [10] or awk script [11], but there was 2 or 3 AM and I just made it work.

screen `socat -d -d ./Thesis.ttyS0 PTY 2>&1 | tee /tmp/x &>/dev/null & grep '.*N\ PTY\ is\ ' /tmp/x | sed -e 's/.*N\ PTY\ is\ //g'`

So the above line does the following:

1. Map the socket
2. Two choices here, I chose the first one because was faster but YMMV.
1.  Use 2>&1 | tee /tmp/x &>/dev/null and redirect the output to a file and optionally to the terminal (I did it for debugging but wasn't interested in keeping it).

socat -d -d ./Thesis.ttyS0 PTY 2>&1 | tee /tmp/x &>/dev/null & grep '.*N\ PTY\ is\ ' /tmp/x | sed -e 's/.*N\ PTY\ is\ //g'


1. Use &> /tmp/x & sleep 1 to give time to socat to write in the file and then read it to get the device name.

socat -d -d ./Thesis.ttyS0 PTY &> /tmp/x & sleep 1 ; grep '.*N\ PTY\ is\ ' /tmp/x | sed -e 's/.*N\ PTY\ is\ //g' 


3. Attach to the screen
   screen `all the above thing in backquotes to execute it before`

Again the final one-liner

screen `socat -d -d ./Thesis.ttyS0 PTY 2>&1 | tee /tmp/x &>/dev/null & grep '.*N\ PTY\ is\ ' /tmp/x | sed -e 's/.*N\ PTY\ is\ //g'`

--

  = ^ . ^ =

[1] http://wiki.networksecuritytoolkit.org/nstwiki/index.php/Console_Output_and_Serial_Terminals

[2] http://fixunix.com/slackware/537945-nc-does-not-support-unix-domain-socket.html

[3] http://linux.die.net/man/1/screen

[4] http://www.dest-unreach.org/socat/

[5] http://www.linuxsmiths.com/blog/?p=312

[6] http://blackmagic02881.wordpress.com/2007/02/05/linux-serial-console-how-to-with-vmware-server/

[7] http://thewayeye.net/2009/december/4/connecting-virtual-machines-serial-console-os-x-and-vmware-fusion[8] http://communities.vmware.com/thread/33528

[9] http://communities.vmware.com/thread/28508

[11] http://linux.die.net/man/1/sed 

[10] http://linux.die.net/man/1/awk

Update: I ported the script to make it work with Linux, check out the new post for details and also the @Github gist.
--
= ^ . ^ =


Boot grub through EFI in a MacBook

Make my mac boot through the EFI Boot interface

1. Make a small HFS+ partition (700MB could do it). I called it "EFI Boot".
2. mkdir -p /Volumes/EFI\ Boot/efi/boot
3. Compile your grub-efi following the instructions posted here [1]
 * It depends on the efi architecture: bootx64.efi is for a 64-bit EFI and bootx32.efi is for the 32 bit Implementation.

After that you reboot and press the 'option' key when the apple chime is heard and there will be a disk called "EFI Boot", simply select it and you will be booting into grub.

[1] http://wiki.osdev.org/GRUB#Build_Grub_EFI_binary_.28bootx64.efi.29
--
  = ^ . ^ =

OpenBSD - Check hosts alive

#!/bin/sh


# Check all hosts within the network
# BSD license


PING=/sbin/ping
SEQ=gseq


NET=192.168.2
ME=192.168.0.2


i=1;
while [ $i -le 254 ] ;
do
  $PING -v -D -s 8 -t 1 -w 1 -c 1 -I $ME $NET.$i 1>/dev/null
  printf "$?"
  i=`expr $i + 1` ;
done


printf "\n"

Thanks to this site [1] for the while loop

[1] http://www.linuxmisc.com/27-linux-on-alpha/9fdb61f03bee119e.htm

--

  = ^ . ^ =

Round Robin DNS redirect in OpenBSD

Redirect all dns requests to a pool of trusted DNS servers (in this case OpenDNS)

Configuration in /etc/pf.conf

dns_servers = "{ 208.67.222.222 208.67.220.220 }"

pass in quick on $inside_if proto udp from <allowed> to any port 53 rdr-to $dns_servers round-robin 

You might also want to add sticky-address to keep asking to one dns server [1] (not my case)

The result when analyzing on the internal interface

# tcpdump -ni rtw0 'port 53 and host ( 8.8.8.8 )'

17:44:46.295443 192.168.127.36.37796 > 8.8.8.8.53: 22244+ A? toneji.to. (31)

17:44:46.384365 8.8.8.8.53 > 192.168.127.36.37796: 22244 1/0/0 A 67.215.65.132 (47)

The result when analyzing on the external interface

# tcpdump -ni xl0 'port 53 and host ( 208.67.220.220 or 208.67.222.222 )'

17:44:46.295561 10.0.2.2.59847 > 208.67.220.220.53: 22244+ A? toneji.to. (31)

17:44:46.384302 208.67.220.220.53 > 10.0.2.2.59847: 22244 1/0/0 A 67.215.65.132 (47) (DF)

This means that *any* DNS request will be forwarded to our trusted DNS servers (Thanks OpenDNS)

References:

[1]  http://www.openbsd.org/faq/pf/pools.html

--

  = ^ . ^ =

make random-fractal

% cat Makefile

BASH=/bin/bash
ECHO=/bin/echo
SLEEP=/bin/sleep
BC=/usr/bin/bc
PRINTF=/usr/bin/printf

random-fractal: shell
        while ${SLEEP} 0.1 ; \
        do \
          if [ `${ECHO} $$RANDOM % 2 | ${BC}` -eq 0 ] ; \
          then \
            ${PRINTF} " " ; \
          else \
            ${PRINTF} "█" ; \
          fi ; \
        done ;
shell:
        $(eval SHELL := ${BASH})

% make

--

  = ^ . ^ =

Zero or one?. Pick

% while sleep 0.1 ; do printf "$(($RANDOM % 2))" ; done ;

--

  = ^ . ^ =

SQL TEST

CREATE DATABASE _ ;

CREATE TABLE _._ (_ int(1) not null primary key);

INSERT INTO _._(_) VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);

SELECT * FROM _._;

DROP TABLE _._ ;

DROP DATABASE _;

--

  = ^ . ^ =

Delete temp files and directories

% while sleep 0.1 ; do if [ $(($RANDOM % 2)) -eq 0 ] ; then rmdir -v `mktemp -d` ; else rm -v `mktemp` ; fi ; done ; 


--
  = ^ . ^ =

show my ip address

#!/bin/sh

IP=/bin/ip

SED=/bin/sed

CUT=/usr/bin/cut

IF=en0

if [ ! -z ${1} ]

then

  IF=${1}

fi

$IP addr show dev $IF | $SED -n 3p | $SED -e 's/\ \+/\ /g' -e 's/\/.*$//g' | $CUT -d ' ' -f 3

--

  = ^ . ^ =

IPv4 and IPv6 SOCKS proxy

$ cat Makefile
XTERM=/usr/bin/xterm
SSH=/usr/bin/ssh
GEOMETRY=169x39-0-0
IPv6_LOCALHOST=::1
PROXY_PORT=1080
SSH_PORT?=22
IPv4_PROXY=127.128.129.130
IPv6_PROXY=${IPv6_LOCALHOST}
IPv4_REMOTE?=127.126.125.124
IPv6_REMOTE=${IPv6_LOCALHOST}
IPv4_BIND=127.127.127.127
IPv6_BIND=${IPv6_LOCALHOST}

proxy:
        ${XTERM} -geometry ${GEOMETRY} -iconic -T "IPv4 proxy" \
          -e ${SSH} -v -x -n -N -b $(IPv4_BIND) -p ${SSH_PORT} \
          -D ${IPv4_PROXY}:${PROXY_PORT} ${IPv4_REMOTE} &
        ${XTERM} -geometry ${GEOMETRY} -iconic -T "IPv6 proxy" \
         -e ${SSH} -v -x -n -N -b $(IPv6_BIND) -p ${SSH_PORT} \
         -D [${IPv6_PROXY}]:${PROXY_PORT} ${IPv6_REMOTE} &

$ make proxy

--

  = ^ . ^ =

make my life hard

% cat Makefile
# = ^ . ^ =
_:
        make -f $(PWD)/Makefile &


% make


--
  = ^ . ^ =

Keep the robots out

Keep the robots out of your website

% cat $DocumentRoot/robots.txt

User-agent: *

Disallow: /

--

= ^ . ^ =

Apache httpd identification strings

The problem...

$ curl --verbose --user-agent "= ^ . ^ =" "http://localhost:80/info.php" > /dev/null

* About to connect() to localhost port 80 (#0)

* Trying 127.0.0.1... connected

* Connected to localhost (127.0.0.1) port 80 (#0)

> GET /info.php HTTP/1.1

> User-Agent: = ^ . ^ =

> Host: localhost

> Accept: */*

>

< HTTP/1.1 200 OK

< Date: Thu, 19 Jan 2012 23:59:59 GMT

< Server: Apache/2.2.16 (Debian) PHP/5.3.3-7+squeeze3 with Suhosin-Patch mod_ssl/2.2.16 OpenSSL/0.9.8o

< X-Powered-By: PHP/5.3.3-7+squeeze3

< Vary: Accept-Encoding

< Transfer-Encoding: chunked

< Content-Type: text/html

<

{ [data not shown]

* Connection #0 to host localhost left intact

* Closing connection #0

The solution...

/etc/apache2/conf.d/security

• ServerTokens Prod
• ServerSignature Off

/etc/php5/apache2/php.ini

• expose_php = Off

/etc/init.d/apache2 restart

$ curl --verbose --user-agent "= ^ . ^ =" "http://localhost:80/info.php" > /dev/null

* About to connect() to localhost port 80 (#0)

* Trying 127.0.0.1... connected

* Connected to localhost (127.0.0.1) port 80 (#0)

> GET /info.php HTTP/1.1

> User-Agent: = ^ . ^ =

> Host: localhost

> Accept: */*

>

< HTTP/1.1 200 OK

< Date: Thu, 20 Jan 2012 00:00:00 GMT

< Server: Apache

< Vary: Accept-Encoding

< Transfer-Encoding: chunked

< Content-Type: text/html

<

{ [data not shown]

* Connection #0 to host localhost left intact

* Closing connection #0

# rm -v /var/www/info.php

--

= ^ . ^ =