Friday 30 March 2012

HTTP POST Test site


Use this website if you would like to test your HTTP POST scripts/programs without having to write a server-side test solution.

http://www.posttestserver.com/

Using curl to download files


Example
Code Snippet
  1. curl -C - -O http://www.mirror.com/path/to/NeoOffice-Patch.dmg
End of Code Snippet

Parameters Used
-o/--output Write output to instead of stdout (Outputs the file to the window by default)
-C/--continue-at Resumed transfer offset

Unix - I don't know what kind of terminal you are on - all I have is 'xterm-color'.


If you receive the following error, typically when using text editors like vi, then you need to change your Terminal (Environment variable: TERM)

xterm-color: Unknown terminal type
I don't know what kind of terminal you are on - all I have is 'xterm-color'.


Depending on which shell your using, set the following...

csh shell
Code Snippet
  1. setenv TERM xterm
End of Code Snippet

bash
Code Snippet
  1. export TERM='xterm'
End of Code Snippet

xterm is an example of a terminal emulator. You can use whichever you would like, it's simply a program that emulates a video terminal within some display architecture. Programs gather information from the terminal your currently using (Set in TERM) to aide the display of the content/program.

Here is a good list of various terminals you can try: List of Terminals

Unix Shells


csh (C-Shell)
The C shell is a command processor that's typically run in a text window, allowing the user to type commands which cause actions. The C shell can also read commands from a file, called a script. Like all Unix shells, it supports filename wildcarding, piping, here documents, command substitution, variables and control structures for condition-testing and iteration. What differentiated the C shell, especially in the 1980s, were its interactive features and overall style. Its new features made it easier and faster to use. The overall style of the language looked more like C and was seen as more readable.

tcsh (TENEX C-Shell)
On many systems such as Mac OS X and Red Hat Linux csh is actually tcsh, an improved version of csh. One file containing the tcsh executable has links to it as both "csh" and "tcsh" so that either name refers to the same improved version of the C shell.

On Ubuntu and Solaris machines, there are two different packages: csh and tcsh. The former is a based on the original BSD version of csh and the later is the improved tcsh. To enable the improved version of csh (tcsh), just type 'tcsh'... You will now notice you have access to command history, working arrows etc...

Other shells exist, such as: bash, sh, ksh, zsh, rc etc... Each having their own unique advantages, however, the common shell principles still apply.



Shell startup Scripts (.files)
When you enter a shell (by logging in for example) or switch to another shell (su or executing a different shell), certain shell startup scripts will be executed. It can become confusing as to which shells use which files (I.e. .cshrc, .profile etc etc).

The following as a list of which shells use which files...

csh
Some versions have system-wide .cshrc and .login files. Every
version puts them in different places.

Start-up (in this order):
.cshrc - always; unless the -f option is used.
.login - login shells.

Upon termination:
.logout - login shells.

Others:
.history - saves the history (based on $savehist).

tcsh
Start-up (in this order):
/etc/csh.cshrc - always.
/etc/csh.login - login shells.
.tcshrc - always.
.cshrc - if no .tcshrc was present.
.login - login shells

Upon termination:
.logout - login shells.

Others:
.history - saves the history (based on $savehist).
.cshdirs - saves the directory stack.

sh
Start-up (in this order):
/etc/profile - login shells.
.profile - login shells.

Upon termination:
any command (or script) specified using the command:
trap "command" 0

ksh
Start-up (in this order):
/etc/profile - login shells.
.profile - login shells; unless the -p option is used.
$ENV - always, if it is set; unless the -p option is used.
/etc/suid_profile - when the -p option is used.

Upon termination:
any command (or script) specified using the command:
trap "command" 0

bash
Start-up (in this order):
/etc/profile - login shells.
.bash_profile - login shells.
.profile - login if no .bash_profile is present.
.bashrc - interactive non-login shells.
$ENV - always, if it is set.

Upon termination:
.bash_logout - login shells.

Others:
.inputrc - Readline initialization.

zsh
Start-up (in this order):
.zshenv - always, unless -f is specified.
.zprofile - login shells.
.zshrc - interactive shells, unless -f is specified.
.zlogin - login shells.

Upon termination:
.zlogout - login shells.

rc
Start-up:
.rcrc - login shells

pkg-get and pkgutil package management for Solaris/SunOS


pkg-get
Pkg-get is an open-source, Sun-approved software installation and management tool for Sun Solaris. You can use pkg-get to install a variety of pre-compiled software without having to worry about version compatibility, patch requirements, libraries, or any other prerequisites.

pkgutil
Pkgutil, written in Perl and licensed under GPL, is a tool to make installation of packages in Solaris easier. It handles package dependencies so all required packages are installed before the desired package automatically. This is based on functionality present in apt-get (Debian Linux).

Which tool?
Both tools offer a similar deal... They aide package management on SunOS, they work out dependencies for packages and versioning is controlled by the package manager. I personally use pkgutil, simply because The Open Community Software Project (OpenCSW) has now deprecated support for pkg-get users (This is a large database of free Solaris packages - Note: There are many others you can use with pkg-get)

OpenCSW/Blastwave
The Open Community Software Project (OpenCSW) is an open-source project providing Solaris binary packages of freely available or open-source software. This was previously hosted at Blastwave. The project was originally Blastwave, but this was forked to created The OpenCSW.
OpenCSW provides packages for Solaris 9 and 10, for 32 and 64-bit, x86 and SPARC architectures. Solaris 8 is no longer a 1st-class supported OS[2]: however, there still exists a legacy Solaris 8 archive, which occasionally gets updates.

This isn't the only place to retrieve Solaris packages, but with a nice large database (3500+ packages at time of writing), its quite beneficial to consume.

Getting Started with pkgutil
http://www.blastwave.org/jir/blastwave.fam


Mirrors
OpenCSW/Blastwave do not directly host packages... A number of mirror sites are available to get these, but you must set this in your pkgutil config file. This is typically located here: /etc/opt/csw/pkgutil.conf

To view a list of mirrors, visit The OpenCSW Mirror Page

When selecting a mirror, you must add '/stable' or '/unstable' to the end of the URL. Unstable packages will be more up to date, but may have bugs present. Stable, on the other hand, will have older packages but more robust.

Open your configuration file for editing and alter the "mirror" section. The below example is a stable UK mirror site.
Code Snippet
  1. mirror=http://www.grangefields.co.uk/mirrors/csw/stable/
End of Code Snippet

Save the file and attempt to update the pkgutil catelog (Basically an index of all availalbe packages for a mirror).

Update the pkgutil catelog
Code Snippet
  1. pkgutil -U
End of Code Snippet

Now the catelog is up to date, we can begin to use pkgutil to download and install packages.


Things to note
Ensure wget is in your system PATH (which wget)... pkgutil ships with it's own version, but most distributions of Solaris have a version located here: /usr/sfw/bin

Update pkgutil using the following command
Code Snippet
  1. pkgutil -u pkgutil
End of Code Snippet

Install packages using the following (I.e. bash)
Code Snippet
  1. pkgutil -i bash
End of Code Snippet

Thursday 22 March 2012

Download older versions of Oracle products


Welcome to the Oracle Software Delivery Cloud

Wednesday 21 March 2012

Rar Command Line Usage Examples


Rar a single file/directory
Code Snippet
  1. rar a NameOfYourArchive.rar /path/to/file
End of Code Snippet

Rar multiple files
Code Snippet
  1. rar a NameOfYourArchive.rar /path/to/files/*
End of Code Snippet

Unrar a *.rar compressed file
Code Snippet
  1. unrar x NameOfYourArchive.rar
End of Code Snippet

List the contents of a rar file without uncompressing it
Code Snippet
  1. unrar l NameOfYourArchive.rar
End of Code Snippet

Restarting and managing services on Mac OS X


Method 1

Searching running processes to get he PID, then killing it (The service will automatically restart).

Example
Code Snippet
  1. sudo ps -A | grep -i ssh
End of Code Snippet

23450 ?? S 0:00.12 /usr/sbin/sshd -i
23455 ?? S 0:00.03 /usr/sbin/sshd -i
23649 p8 S+ 0:00.00 grep -i ssh


Now use 'kill' on the PID...
Code Snippet
  1. sudo kill 23450
End of Code Snippet



Method 2

launchd is a unified, open-source service management framework for starting, stopping and managing daemons, applications, processes, and scripts. Written and designed by Dave Zarzycki at Apple, it was introduced with Mac OS X Tiger and is licensed under the Apache License.

We will be using launchctl, a subset of this framework to manage services...

List all services
Code Snippet
  1. launchctl list
End of Code Snippet

We can then use this list to identify the names of our services.

Here's an example with SSH...

Stopping a service
Code Snippet
  1. launchctl stop com.openssh.sshd
End of Code Snippet

Starting a service
Code Snippet
  1. launchctl start com.openssh.sshd
End of Code Snippet

How to enable X11 Forwarding with SSH on Mac OS X


The aim of this blog post is to discuss how to display an X window from a target machine on our local system.

A good example here would be the Oracle installer for linux... when the installer is executed, it utilises the windowing system configured to display the installer via. SSH. Lets see how we can set this up...


What is the X windowing system (or X11)
The X window system (commonly X Window System or X11, based on its current major version being 11) is a computer software system and network protocol that provides a basis for graphical user interfaces (GUIs) and rich input device capability for networked computers.

Advantages of using SSH for X11 Traffic
- No problems with firewalls, as long as they let SSH pass through (No need to allow access to tcp-port 6000+)
- Communication is encrypted


Note to other OS users [Windows etc]
As long as your software/OS implements the X windowing system, you will be able to display X windows. For windows users, I recommend PuTTY


Mac OS X Configuration

Enable X11 Forwarding in sshd_config
In order to allow X11 forwarding to pass through SSH, we need to enable X11 Forwarding in our config for SSH. This is located in a file called sshd_config and found typically here: /etc/sshd_config or /etc/ssh/sshd_config

Uncomment out the line "# X11Forwarding no" and set it to "yes" (If this hasn't been done already. You will be left with this configuration...

Code Snippet
  1. X11Forwarding yes
End of Code Snippet

Save the file, and restart the sshd service...


Add your target system to the access control list
So our windowing system on our local machine has an access control list (We cant let anybody in as and when we please!)... So we need to allow our target system access to our windowing system.

Code Snippet
  1. xhost fullyqualified.server.name
End of Code Snippet

You will then see this message to confirm...
fullyqualified.server.name being added to access control list


SSH onto our target machine (with the apps using the windowing system. I.e. Oracle Installer)
We can now SSH onto our target machine...

Code Snippet
  1. ssh -X user@fullyqualified.server.name
End of Code Snippet


Ensure the DISPLAY environment variable is configured correctly
On the machine we just SSH'd onto, we need to make sure the DISPLAY environment variable is configured correctly. Our apps will use this to find out where our target machine that supports the X windowing system is located.

Code Snippet
  1. DISPLAY=fullyqualifiedaddressforourhostmachine:0.0
End of Code Snippet

Ensure this has been set correctly...
Code Snippet
  1. echo $DISPLAY
End of Code Snippet

Now lets test to see if our X apps can use our host windowing system via. SSH...
Code Snippet
  1. xclock &
End of Code Snippet

If you do not have xclock, or maybe your trying to execute an installer for Oracle or something, then try and execute the installer.


Gotchas
- Do not set the DISPLAY variable in any login scripts (I.e. bashrc, bash_profile etc...)

Friday 9 March 2012

Python - POST HTTP multipart form (Using standard urllib libraries)


The Python script in this example utilises standard Python libraries to POST a form with content type multipart/form-data over HTTP (or quite simply, a multipart form).

The content type "multipart/form-data" should be used for submitting forms that contain files, non-ASCII data, and binary data.

I won't delve deep into the details on multipart forms, but if you would like a good explanation, read the official W3C document. One thing if doesn't mention on there, is that the boundary can be between 1 and 70 characters long, consisting of alphanumeric, and the punctuation you see in the list. Spaces are allowed except at the end.

Code Snippet
  1.     import itertools
  2.     import mimetools
  3.     import mimetypes
  4.     from cStringIO import StringIO
  5.     import urllib
  6.     import urllib2
  7.      
  8.     class MultiPartForm(object):
  9.         """Accumulate the data to be used when posting a form."""
  10.      
  11.         def __init__(self):
  12.             self.form_fields = []
  13.             self.files = []
  14.            
  15.             # Generate unique boundary string with the format: hostipaddr.uid.pid.timestamp.random
  16.             self.boundary = mimetools.choose_boundary()
  17.             return
  18.        
  19.         def get_content_type(self):
  20.             return 'multipart/form-data; boundary=%s' % self.boundary
  21.      
  22.         def add_field(self, name, value):
  23.             """Add a simple field to the form data."""
  24.             self.form_fields.append((name, value))
  25.             return
  26.      
  27.         def add_file(self, fieldname, filename, fileHandle, mimetype=None):
  28.             """Add a file to be uploaded."""
  29.             body = fileHandle.read()
  30.             if mimetype is None:
  31.                 mimetype = mimetypes.guess_type(filename)[0] or 'application/octet-stream'
  32.             self.files.append((fieldname, filename, mimetype, body))
  33.             return
  34.        
  35.         def __str__(self):
  36.             """Return a string representing the form data, including attached files."""
  37.             # Build a list of lists, each containing "lines" of the
  38.             # request.  Each part is separated by a boundary string.
  39.             # Once the list is built, return a string where each
  40.             # line is separated by '\r\n'.  
  41.             parts = []
  42.             part_boundary = '--' + self.boundary
  43.            
  44.             # Add the form fields
  45.             parts.extend(
  46.                 [ part_boundary,
  47.                   'Content-Disposition: form-data; name="%s"' % name,
  48.                   '',
  49.                   value,
  50.                 ]
  51.                 for name, value in self.form_fields
  52.                 )
  53.            
  54.             # Add the files to upload
  55.             parts.extend(
  56.                 [ part_boundary,
  57.                   'Content-Disposition: file; name="%s"; filename="%s"' % \
  58.                      (field_name, filename),
  59.                   'Content-Type: %s' % content_type,
  60.                   '',
  61.                   body,
  62.                 ]
  63.                 for field_name, filename, content_type, body in self.files
  64.                 )
  65.            
  66.             # Flatten the list and add closing boundary marker,
  67.             # then return CR+LF separated data
  68.             flattened = list(itertools.chain(*parts))
  69.             flattened.append('--' + self.boundary + '--')
  70.             flattened.append('')
  71.             return '\r\n'.join(flattened)
  72.      
  73.      
  74.      
  75.      
  76.      
  77.     if __name__ == '__main__':
  78.      
  79.         # Create the form with simple fields
  80.         form = MultiPartForm()
  81.        
  82.        
  83.         """
  84.            Either specify a source, file or a url...
  85.            The system will take the first one respectively if all three have content.
  86.            This example uses a dummy file called test.sql, with a string of content.
  87.       """
  88.        
  89.         # No source specified
  90.         form.add_field('source', '')
  91.        
  92.         # REAL FILE EXAMPLE
  93.         # fileHandle = open ('test.sql', 'r')
  94.         # form.add_file('file', 'test.sql', fileHandle)
  95.         # fileHandle.close()
  96.        
  97.         # Use dummy file
  98.         form.add_file('file', 'test.sql', fileHandle=StringIO('FILE CONTENTS'))
  99.        
  100.         # No URL specified
  101.         form.add_field('url', '')
  102.        
  103.        
  104.        
  105.         # Add remaining required parameters
  106.         form.add_field('language', 'php')
  107.         form.add_field('line_numbers', '2')
  108.         form.add_field('word_wrap', 'on')
  109.         form.add_field('tab_width', '8')
  110.         form.add_field('highlight_keywords', 'on')
  111.         form.add_field('default_color', '0000bb')
  112.         form.add_field('keyword_colors[1]', 'DEFAULT')
  113.         form.add_field('keyword_colors[2]', 'DEFAULT')
  114.         form.add_field('keyword_colors[3]', 'DEFAULT')
  115.         form.add_field('keyword_colors[4]', 'DEFAULT')
  116.         form.add_field('comments_color', 'DEFAULT')
  117.         form.add_field('escaped_chars_color', 'DEFAULT')
  118.         form.add_field('brackets_color', 'DEFAULT')
  119.         form.add_field('strings_color', 'DEFAULT')
  120.         form.add_field('numbers_color', 'DEFAULT')
  121.         form.add_field('methods_color', 'DEFAULT')
  122.      
  123.         # Build the request
  124.         request = urllib2.Request('http://qbnz.com/highlighter/php_highlighter.php')
  125.        
  126.         # Hi i'm FireFox 5
  127.         request.add_header('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11')
  128.        
  129.         # We are required to specify the content type and length...
  130.         body = str(form)
  131.         request.add_header('Content-type', form.get_content_type())
  132.         request.add_header('Content-length', len(body))
  133.         request.add_data(body)
  134.      
  135.             # Take a look at what we are sending... useful to match up against the required request
  136.         print
  137.         print 'OUTGOING DATA:'
  138.         print request.get_data()
  139.      
  140.             # View servers response
  141.         print
  142.         print 'SERVER RESPONSE:'
  143.         print urllib2.urlopen(request).read()
End of Code Snippet

Thursday 8 March 2012

GeSHi - Generic Programming Code Syntax Highlighter


Code syntax highlighter written in PHP
http://qbnz.com/highlighter/

You can easily interface with blogger/blogspot by using the demo website and copying out the CSS for your chosen language. Each language will have a different CSS stylesheet, so if you have to tweak one language to fit your blogspot template, then you will need to apply this to the others.

Examples

Default code block stylings
This is a standard code block [<code>This is a standard code block</code>]


Using GeSHi stylings
Code Snippet
  1. This is an example code snippet
  2. # Here is a comment, btw we are using Bash code highlighting
  3.  
  4.   if [ $THIS -eq 999 ]; then
  5.     eval "lets evaluate something here"
  6.   fi
  7.  
  8. # Have fun!
End of snippet

Line numbers are excluded when copying a code snippet to the clipboard also, which is another good thing!

IMPORTANT UPDATE

With BlogSpot/Blogger, if you add a new post and give it a Label with the same name as a chosen language (I.e. Add Python code to a post, and assign a Python Label/Categroy to the post (which is quite common))... Your blog post will appear garbled. This is because Blogger assigns the label to a class and this will take on your CSS styles from GeSHi

Here is a link to a CSS file I have used with most of the languages I talk about in. It support all the default settings, but with word wrap and line numbers both ON. I have prefixed all CSS classes with "sh_" to avoid this problem. Remember to change the class of the main DIV when taking the styles from GeSHi.

Download link for modified CSS

Unix Shell Script - Get Tomorrows Date


This code snippet will calculate tomorrows date in the format DDMMYYYY. It takes into account correct monthly overlaps and leap years etc...


Code Snippet
  1.   # default output format, change as necessary
  2.   defaultof="%Y%m%d"
  3.  
  4.   # check for input format, else use default format,
  5.   # refer to 'man date' for help on format
  6.   # script only supports %Y %m %d at the moment
  7.   of=$defaultof
  8.   [ $# -eq 1 ] && of="$1"
  9.  
  10.   # get today's date
  11.   eval "`date +'y=%Y m=%m d=%d'`"
  12.  
  13.   #check for max number of days in current month
  14.   days=31
  15.   if [ $m -eq 4 ] || [ $m -eq 6 ] || [ $m -eq 9 ] || [ $m -eq 11 ] ; then
  16.     days=30
  17.   fi
  18.   # check for leap year if feb
  19.   if [ $m -eq 2 ]; then
  20.     days=28
  21.     leap1=`expr $y % 4`
  22.     leap2=`expr $y % 100`
  23.     leap3=`expr $y % 400`
  24.     if [ $leap1 -eq 0 ] ; then
  25.       if [ $leap2 -gt 0 ] || [ $leap3 -eq 0 ] ; then
  26.         days=29
  27.       fi
  28.     fi
  29.   fi
  30.  
  31.   # increment date
  32.   if [ $d -eq $days ]; then
  33.     d=1
  34.     m=`expr $m + 1`
  35.     if [ $m -eq 13 ]; then
  36.       m=1
  37.       y=`expr $y + 1`
  38.     fi
  39.   else
  40.     d=`expr $d + 1`
  41.   fi
  42.  
  43.   #Solaris date does not accept -d
  44.   #date -d "$y-$m-$d" +"$of"
  45.  
  46.   eval "y=`expr $y + 0` m=`expr $m + 0` d=`expr $d + 0`"
  47.   eval "y=`printf "%04d" $y` m=`printf "%02d" $m` d=`printf "%02d" $d`"
  48.   tomorrowsDate="${d}${m}${y}"
End of snippet

Wednesday 7 March 2012

iptables/Netfilter - Command line overview


iptables/netfilter
Most distributions of linux have an inbuilt firewall, commonly refered to as iptables. In actuality and more accuratly, it is iptables/netfilter. iptables sits in user space where the user can interact and manage rulesets. Netfilter is a kernal module, built into the kernal, that actually does the filtering

iptables is used to set up, maintain, and inspect the tables of IPv4 packet filter rules in the Linux kernel. IPv6 addresses can be managed with ip6tables.

A number of GUIs can be used to manage the process, but often lack the flexibility of the command line. Lets take a look at the process in more detail...

Chains
With a new installation, three pre-defined chains exist (INPUT, OUTPUT and FORWARD)... You can add new chains, remove them and rename them. They simply serve as a container for holding rules. You will refer to the name of your chain when making new rules and managing existing ones.

Let's assume we stuck with the pre-defined chains...

INPUT - All packets destined for the host computer (I.e. keeping people/processes out!)
OUTPUT - All packets originating from the host computer.
FORWARD - All packets neither destined for nor originating from the host computer, but passing through (routed by) the host computer. This chain is used if you are using your computer as a router.


The ordering of rules in a chain does matter. A packet is checked against each rule in turn. Once a packet matches a specific rule (Starting at rule 1), then the action is taken on that packet (ACCEPT, DROP etc..), all other rules below this are ignored. If a packet makes it all the way down the rule chain with no matches, then the default action for that chain is taken. This is referred to as the default policy and may be set to either ACCEPT or DROP the packet.


Let's see a live example...


To display a list of current rules, execute the following command:
Code Snippet
  1. iptables --list
End of Code Snippet


This will display the following...

Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
ACCEPT icmp -- anywhere anywhere icmp any
ACCEPT esp -- anywhere anywhere
ACCEPT ah -- anywhere anywhere
ACCEPT udp -- anywhere 224.0.0.251 udp dpt:mdns
ACCEPT udp -- anywhere anywhere udp dpt:ipp
ACCEPT tcp -- anywhere anywhere tcp dpt:ipp
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited


Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination


You can see that the default policy is to REJECT the packet on all protocols, from any source and to any destination.

You can view all of the commands for iptables by executing the following command:

Code Snippet
  1. iptables --help
End of Code Snippet

In the examples section below, I have listed some common commands for managing firewall rules. Below each, discusses the parameters used. I have used a range of commands which allow me to explain the majority of the accepted values in detail...


EXAMPLES


List all IP chains with line numbers

Code Snippet
  1. iptables --list --line-numbers
End of Code Snippet


- Lists all rules for all chains, also echo's line numbers top the left (This allows us to insert rules at certain positions)

Insert a rule at position 13 (Chain: INPUT, Protocol: TCP, Port 115)
Code Snippet
  1. iptables --insert INPUT 13 -m state --state NEW -p tcp --dport 115 -j ACCEPT
End of Code Snippet


States
- Here we're using the -m switch to load a module (state). The state module is able to examine the state of a packet and determine if it is NEW, ESTABLISHED or RELATED
- NEW refers to incoming packets that are new incoming connections that weren't initiated by the host system.
- ESTABLISHED and RELATED refers to incoming packets that are part of an already established connection or related to and already established connection.

-j ACCEPT
-j (jump) to the target action for packets matching the rule - in this case ACCEPT.


Append a rule to the bottom of a chosen chain (In this case the INPUT chain) for an interface [localhost])
Code Snippet
  1. iptables --append INPUT -i lo -j ACCEPT
End of Code Snippet


- 'lo' represents localhost (127.0.0.1)... This is generally required as many software applications expect to be able to communicate with the localhost adaptor.

Remove all rules in all chains
Code Snippet
  1. iptables --flush
End of Code Snippet


Remove all rules in a chain
Code Snippet
  1. iptables --flush [Chain Name]
End of Code Snippet

Remove a rule (Remove a rule at position 13)
Code Snippet
  1. iptables --delete INPUT 13
End of Code Snippet


Saving Rules (Persisting changing on system reboot)

Firewall changes need to be saved in order to be persisted on system reboot. We can achieve this by using the following command...

Code Snippet
  1. service iptables save
End of Code Snippet

Saving firewall rules to /etc/sysconfig/iptables: [ OK ]


This executes the iptables init script, which runs /sbin/iptables-save and writes the current iptables configuration to /etc/sysconfig/iptables. Upon reboot, the iptables init script reapplies the rules saved in /etc/sysconfig/iptables by using the /sbin/iptables-restore command.



References and more info
http://wiki.centos.org/HowTos/Network/IPTables

Tuesday 6 March 2012

tcpdump localhost


Capture packets on localhost from localhost

Code Snippet
  1. sudo tcpdump -i lo
End of Code Snippet

Monday 5 March 2012

WSDL (Web Service Description Language) - Complete Overview


The aim of this article is to describe what WSDL is, how we can utilise the technology from a development perspective and dive into technical detail on how the technology fits together and what is possible. This is an initial draft, but I will be updating it casually over the next few weeks. I have an example Math WSDL file provided by Microsoft throughout this article (Full appendix at the bottom of this post).


WSDL 1.1 (Web Service Description Language)

A WSDL file is an XML file which describes its associated web service. The WSDL XML file describes: where the web service is located, the methods available, parameters required and their return types. WSDLs are platform in-specific and can be consumed from any accessible environment, framework and/or language (Mac/Windows/Linux, Web Apps/Services, C#/C++/Python/Perl etc...)

Using the WSDL file, a consumer has enough information to query the underlying web services using one or many of it's associated operations. Most languages/frameworks have built-in or third party libraries available that can generate proxy classes based on the WSDL file (Another layer in front of the WSDL/Service you can query against using your chosen language. I.e. wsdl.exe for .NET or WSDL2Java for Java). This is only an option to make things easier, and is by no means mandatory.



Version History

WSDL 1.0 was introduced in 2000 and was developed through a collaboration by Microsoft, IBM and Ariba. The language was formalized in 2001 as version 1.1.

WSDL 2.0 became a W3C recommendation on June 2007. WSDL 1.2 was renamed to WSDL 2.0 because it has substantial differences from WSDL 1.1. The changes are:

Adding further semantics to the description language
Removal of message constructs
No support for operator overloading
PortTypes renamed to interfaces
Ports renamed to endpoint.
Offers binding to all HTTP Request methods (Not just GET and POST [WSDL 1.1]).. So it enabled better RESTful web services.

Most third-party vendors, however, have not adapted their offerings to support WSDL 2.0. For example, the Web Services Business Process Execution Language (WS-BPEL) uses WSDL 1.1.




Document Structure (One or many of these base four definitions may exist)

Code Snippet
  1. <definitions>
  2.  
  3. <types>
  4.   The data types used by the web service
  5. </types>
  6.  
  7. <message>
  8.   The messages used by the web service
  9. </message>
  10.  
  11. <portType>
  12.   The operations performed by the web service
  13. </portType>
  14.  
  15. <binding>
  16.   The communication protocols used by the web service
  17. </binding>
  18.  
  19. <service>
  20.   Information regarding the service endpoints.
  21. </service>
  22.  
  23. </definitions>
End of Code Snippet



Note:
- A WSDL can contain extension elements. For example, it is also possible to group multiple services into one WSDL. However, the core principles behind the technology remain the same.
- It is common for one or many type, mesaage, portType or binding definitions to exist within one WSDL file.




Port Types (Interfaces in 2.0)

The portType section defines all operations available within the web service. It also defines the input, output and fault messages for each operation (A fault tag can only exist with an output tag). The input/output/fault message names are mapped onto message tag names, which describe the names and types of parameters used.

Example

Code Snippet
  1. <message name="AddMessage">
  2. <part name="parameter" element="ns:Add"/>
  3. </message>
  4. <message name="AddResponseMessage">
  5. <part name="parameter" element="ns:AddResponse"/>
  6. </message>
  7. <message name="AddFaultMessage">
  8. <part name="parameter" type="ns:String"/>
  9. </message>
  10.  
  11.  
  12. <portType name="MathInterface">
  13. <operation name="Add">
  14. <input message="y:AddMessage"/>
  15. <output message="y:AddResponseMessage"/>
  16. <fault message="y:AddFaultMessage"/>
  17. </operation>
  18. <operation name="Subtract">
  19. <input message="y:SubtractMessage"/>
  20. <output message="y:SubtractResponseMessage"/>
  21. <fault  message="y:SubtractFaultMessage"/>
  22. </operation>
  23. <operation name="Multiply">
  24. <input message="y:MultiplyMessage"/>
  25. <output message="y:MultiplyResponseMessage"/>
  26. <fault  message="y:MultiplyFaultMessage"/>
  27. </operation>
  28. <operation name="Divide">
  29. <input message="y:DivideMessage"/>
  30. <output message="y:DivideResponseMessage"/>
  31. <fault  message="y:DivideFaultMessage"/>
  32. </operation>
  33. </portType>
End of Code Snippet


The above example defines a single portType/Interface with four operations (Add, Subtract, Multiply and Divide). These are all Request-response type operations (because they have input and output) that also include faults.

There are four different types of MEPs (Message exchange patterns) that exist when defining operations in WSDL files...


One-way
- Contains only an input message.

Request-response
- Contains an input message, followed by an output message (may contain a fault message)

Solicit-response
- Contains an output message, followed by an input message (may contain a fault message)

Notification
- Contains only an output message.


I will cover messages later in this article, but I have displays the tags here (For the Add operation only) to see how operations are mapped onto messages. AddFaultMessage takes a simple string type, however, complex types (Arrays, Lists, Custom Objects etc..) may exist which require definition in the Types section of the WSDL. Lets have a look at this next...




Types

The types section defines the data types that are used by the service. If the service uses only simple types I.e. Integers, Strings etc... then the types section is not required. WSDL types can also be reused across multiple services. The types section contains one or many schema tags. XML Schema's are the typical choice, however, WSDL 1.1 is not limited to this schema type.


Example

Code Snippet
  1. <types>
  2. <schema targetNamespace="http://example.com/weather.xsd" xmlns="http://www.w3.org/2000/10/XMLSchema">
  3. <xs:complexType name="MathInput">
  4. <xs:sequence>
  5. <xs:element name="x" type="xs:double"/>
  6. <xs:element name="y" type="xs:double"/>
  7. </xs:sequence>
  8. </xs:complexType>
  9. <xs:complexType name="MathOutput">
  10. <xs:sequence>
  11. <xs:element name="result" type="xs:double"/>
  12. </xs:sequence>
  13. </xs:complexType>
  14. <xs:element name="Add" type="MathInput"/>
  15. <xs:element name="AddResponse" type="MathOutput"/>
  16. <xs:element name="Subtract" type="MathInput"/>
  17. <xs:element name="SubtractResponse" type="MathOutput"/>
  18. <xs:element name="Multiply" type="MathInput"/>
  19. <xs:element name="MultiplyResponse" type="MathOutput"/>
  20. <xs:element name="Divide" type="MathInput"/>
  21. <xs:element name="DivideResponse" type="MathOutput"/>
  22. </xs:schema>
  23. </types>
End of Code Snippet



From our previous example, we can see that all four operations map to eight messages (one input, one output, four operations). These 8 messages map on to these eight elements defined in the Types section. Above these elements, you can see two complex types have been defined: MathInput and MathOutput. When contrasting these examples in terms on O-O programming, you can think of a complexType as an Object; and x and y as properties of this object.





Messages (Do not exist in WSDL 2.0 - Operations refer to types directly)

Messages define the input and output of operations. They either map onto an element or a type (Both defined within the Types section). As discussed within the portType section previously, the operation decides which message is used for it's input and output.

Code Snippet
  1. <message name="AddMessage">
  2. <part name="parameter" element="ns:Add"/>
  3. </message>
  4. <message name="AddResponseMessage">
  5. <part name="parameter" element="ns:AddResponse"/>
  6. </message>
  7. <message name="SubtractMessage">
  8. <part name="parameter" element="ns:Subtract"/>
  9. </message>
  10. <message name="SubtractResponseMessage">
  11. <part name="parameter" element="ns:SubtractResponse"/>
  12. </message>
End of Code Snippet



Bindings

Bindings provides specific details on how the portType section will actually be transmitted over the wire. The bindings can be made available via. multiple transports: HTTP GET, HTTP POST and/or SOAP.

You can specify multiple bindings for a single portType. I.e. Making an operation available via GET, POST and SOAP.

The following example illustrates a SOAP/HTTP binding for the MathInterface portType (Defined above)...


Example


Code Snippet
  1. <definitions
  2. xmlns="http://schemas.xmlsoap.org/wsdl/"
  3. xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
  4. xmlns:xs="http://www.w3.org/2001/XMLSchema"
  5. xmlns:y="http://example.org/math/"
  6. xmlns:ns="http://example.org/math/types/"
  7. targetNamespace="http://example.org/math/">
  8. ...
  9. <binding name="MathSoapHttpBinding" type="y:MathInterface">
  10. <soap:binding style="document"
  11. transport="http://schemas.xmlsoap.org/soap/http"/>
  12. <operation name="Add">
  13. <soap:operation soapAction="http://example.org/math/#Add"/>
  14. <input>
  15. <soap:body use="literal"/>
  16. </input>
  17. <output>
  18. <soap:body use="literal"/>
  19. </output>
  20. <fault>
  21. <soap:body use="literal"/>
  22. </fault>
  23. </operation>
  24. ...
  25. </binding>
  26. ...
  27. </definitions>
End of Code Snippet



Binding Info
- The NAME of the binding must be given a unique name. I.e. MathSoapHttpBinding - so it can be referenced from within the WSDL.
- The TYPE of the binding must map onto the name of the portType (Interface WSDL 2.0)

soap:binding Info
- This element indicates that this is a SOAP 1.1 binding.
- The style of service is "document" (See section below on WSDL Styles)
- The transport protocol is HTTP
- The soap:operation element defines the SOAPAction HTTP header value for each operation
- The soap:body element defines how the message parts appear inside of the SOAP Body element (possible values include literal or encoded)




WSDL Styles


A WSDL SOAP binding can be either a Remote Procedure Call (RPC) style binding or a document style binding. A SOAP binding can also have an encoded use or a literal use.

Using document style in SOAP indicates that the body will contain an XML document, and that the message parts specify the XML elements that will be placed there. Using rpc style in SOAP indicates that the body will contain an XML representation of a method call and that the message parts represent the parameters to the method.

This gives you four style/use models:

RPC/encoded
RPC/literal
Document/encoded
Document/literal


The use attribute specifies the encoding that should be used to translate the abstract message parts into a concrete representation. In the case of 'encoded', the abstract definitions are translated into a concrete format by applying the SOAP encoding rules. In the case of 'literal', the abstract type definitions become the concrete definitions themselves (they're 'literal' definitions). In this case, you can simply inspect the XML Schema type definitions to determine the concrete message format. For example, the Add operation for the above document/literal binding looks like this on the wire:

Using literal definitions is much cleaner and easier for the tools to get right. Using the encoding rules has led to significant interoperability problems across toolkits. It also leads to weird situations like not being able to validate the wire-level message against the original schema definition (since it's abstract and not a true representation of the message)

More info on WSDL Styles here: http://www.ibm.com/developerworks/webservices/library/ws-whichwsdl/



Service


The service element defines a collection of endpoints/ports that expose our bindings (Listed above). Each port must be given a name and assigned a binding.

Example

Code Snippet
  1. <definitions
  2. xmlns="http://schemas.xmlsoap.org/wsdl/"
  3. xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
  4. xmlns:xs="http://www.w3.org/2001/XMLSchema"
  5. xmlns:y="http://example.org/math/"
  6. xmlns:ns="http://example.org/math/types/"
  7. targetNamespace="http://example.org/math/">
  8. ...
  9. <service name="MathService">
  10. <port name="MathEndpoint" binding="y:MathSoapHttpBinding">
  11. <soap:address
  12. location="http://localhost/math/math.asmx"/>
  13. </port>
  14. </service>
  15. </definitions>
End of Code Snippet



Full WSDL Example (For all above examples)

Code Snippet
  1. <definitions xmlns="http://schemas.xmlsoap.org/wsdl/"
  2. xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
  3. xmlns:http="http://schemas.xmlsoap.org/wsdl/http/"
  4. xmlns:xs="http://www.w3.org/2001/XMLSchema"
  5. xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
  6. xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/"
  7. xmlns:y="http://example.org/math/"
  8. xmlns:ns="http://example.org/math/types/"
  9. targetNamespace="http://example.org/math/">
  10. <types>
  11. <xs:schema targetNamespace="http://example.org/math/types/"
  12. xmlns="http://example.org/math/types/"
  13. elementFormDefault="unqualified" attributeFormDefault="unqualified">
  14. <xs:complexType name="MathInput">
  15. <xs:sequence>
  16. <xs:element name="x" type="xs:double"/>
  17. <xs:element name="y" type="xs:double"/>
  18. </xs:sequence>
  19. </xs:complexType>
  20. <xs:complexType name="MathOutput">
  21. <xs:sequence>
  22. <xs:element name="result" type="xs:double"/>
  23. </xs:sequence>
  24. </xs:complexType>
  25. <xs:element name="Add" type="MathInput"/>
  26. <xs:element name="AddResponse" type="MathOutput"/>
  27. <xs:element name="Subtract" type="MathInput"/>
  28. <xs:element name="SubtractResponse" type="MathOutput"/>
  29. <xs:element name="Multiply" type="MathInput"/>
  30. <xs:element name="MultiplyResponse" type="MathOutput"/>
  31. <xs:element name="Divide" type="MathInput"/>
  32. <xs:element name="DivideResponse" type="MathOutput"/>
  33. </xs:schema>
  34. </types>
  35. <message name="AddMessage">
  36. <part name="parameters" element="ns:Add"/>
  37. </message>
  38. <message name="AddResponseMessage">
  39. <part name="parameters" element="ns:AddResponse"/>
  40. </message>
  41. <message name="SubtractMessage">
  42. <part name="parameters" element="ns:Subtract"/>
  43. </message>
  44. <message name="SubtractResponseMessage">
  45. <part name="parameters" element="ns:SubtractResponse"/>
  46. </message>
  47. <message name="MultiplyMessage">
  48. <part name="parameters" element="ns:Multiply"/>
  49. </message>
  50. <message name="MultiplyResponseMessage">
  51. <part name="parameters" element="ns:MultiplyResponse"/>
  52. </message>
  53. <message name="DivideMessage">
  54. <part name="parameters" element="ns:Divide"/>
  55. </message>
  56. <message name="DivideResponseMessage">
  57. <part name="parameters" element="ns:DivideResponse"/>
  58. </message>
  59. <portType name="MathInterface">
  60. <operation name="Add">
  61. <input message="y:AddMessage"/>
  62. <output message="y:AddResponseMessage"/>
  63. </operation>
  64. <operation name="Subtract">
  65. <input message="y:SubtractMessage"/>
  66. <output message="y:SubtractResponseMessage"/>
  67. </operation>
  68. <operation name="Multiply">
  69. <input message="y:MultiplyMessage"/>
  70. <output message="y:MultiplyResponseMessage"/>
  71. </operation>
  72. <operation name="Divide">
  73. <input message="y:DivideMessage"/>
  74. <output message="y:DivideResponseMessage"/>
  75. </operation>
  76. </portType>
  77. <binding name="MathSoapHttpBinding" type="y:MathInterface">
  78. <soap:binding style="document"
  79. transport="http://schemas.xmlsoap.org/soap/http"/>
  80. <operation name="Add">
  81. <soap:operation soapAction="http://example.org/math/#Add"/>
  82. <input>
  83. <soap:body use="literal"/>
  84. </input>
  85. <output>
  86. <soap:body use="literal"/>
  87. </output>
  88. </operation>
  89. <operation name="Subtract">
  90. <soap:operation soapAction="http://example.org/math/#Subtract"/>
  91. <input>
  92. <soap:body use="literal"/>
  93. </input>
  94. <output>
  95. <soap:body use="literal"/>
  96. </output>
  97. </operation>
  98. <operation name="Multiply">
  99. <soap:operation soapAction="http://example.org/math/#Multiply"/>
  100. <input>
  101. <soap:body use="literal"/>
  102. </input>
  103. <output>
  104. <soap:body use="literal"/>
  105. </output>
  106. </operation>
  107. <operation name="Divide">
  108. <soap:operation soapAction="http://example.org/math/#Divide"/>
  109. <input>
  110. <soap:body use="literal"/>
  111. </input>
  112. <output>
  113. <soap:body use="literal"/>
  114. </output>
  115. </operation>
  116. </binding>
  117. <service name="MathService">
  118. <port name="MathEndpoint" binding="y:MathSoapHttpBinding">
  119. <soap:address location="http://localhost/math/math.asmx"/>
  120. </port>
  121. </service>
  122. </definitions>
End of Code Snippet




References
http://msdn.microsoft.com/en-us/library/ms996486.aspx



TODO: Add more info on WSDL 2.0