Use this website if you would like to test your HTTP POST scripts/programs without having to write a server-side test solution.
http://www.posttestserver.com/
Technical blog discussing various programming languages, frameworks and paradigms. Code snippets and projects are also provided.
Friday, 30 March 2012
Using curl to download files
Example
Parameters Used
-o/--output Write output to instead of stdout (Outputs the file to the window by default)
-C/--continue-at Resumed transfer offset
Code Snippet
- curl -C - -O http://www.mirror.com/path/to/NeoOffice-Patch.dmg
End of Code Snippet
Parameters Used
-o/--output
-C/--continue-at
Labels:
Unix
Unix - I don't know what kind of terminal you are on - all I have is 'xterm-color'.
If you receive the following error, typically when using text editors like vi, then you need to change your Terminal (Environment variable: TERM)
xterm-color: Unknown terminal type
I don't know what kind of terminal you are on - all I have is 'xterm-color'.
Depending on which shell your using, set the following...
csh shell
bash
xterm is an example of a terminal emulator. You can use whichever you would like, it's simply a program that emulates a video terminal within some display architecture. Programs gather information from the terminal your currently using (Set in TERM) to aide the display of the content/program.
Here is a good list of various terminals you can try: List of Terminals
xterm-color: Unknown terminal type
I don't know what kind of terminal you are on - all I have is 'xterm-color'.
Depending on which shell your using, set the following...
csh shell
Code Snippet
- setenv TERM xterm
End of Code Snippet
bash
Code Snippet
- export TERM='xterm'
End of Code Snippet
xterm is an example of a terminal emulator. You can use whichever you would like, it's simply a program that emulates a video terminal within some display architecture. Programs gather information from the terminal your currently using (Set in TERM) to aide the display of the content/program.
Here is a good list of various terminals you can try: List of Terminals
Labels:
Unix
Unix Shells
csh (C-Shell)
The C shell is a command processor that's typically run in a text window, allowing the user to type commands which cause actions. The C shell can also read commands from a file, called a script. Like all Unix shells, it supports filename wildcarding, piping, here documents, command substitution, variables and control structures for condition-testing and iteration. What differentiated the C shell, especially in the 1980s, were its interactive features and overall style. Its new features made it easier and faster to use. The overall style of the language looked more like C and was seen as more readable.
tcsh (TENEX C-Shell)
On many systems such as Mac OS X and Red Hat Linux csh is actually tcsh, an improved version of csh. One file containing the tcsh executable has links to it as both "csh" and "tcsh" so that either name refers to the same improved version of the C shell.
On Ubuntu and Solaris machines, there are two different packages: csh and tcsh. The former is a based on the original BSD version of csh and the later is the improved tcsh. To enable the improved version of csh (tcsh), just type 'tcsh'... You will now notice you have access to command history, working arrows etc...
Other shells exist, such as: bash, sh, ksh, zsh, rc etc... Each having their own unique advantages, however, the common shell principles still apply.
Shell startup Scripts (.files)
When you enter a shell (by logging in for example) or switch to another shell (su or executing a different shell), certain shell startup scripts will be executed. It can become confusing as to which shells use which files (I.e. .cshrc, .profile etc etc).
The following as a list of which shells use which files...
csh
Some versions have system-wide .cshrc and .login files. Every
version puts them in different places.
Start-up (in this order):
.cshrc - always; unless the -f option is used.
.login - login shells.
Upon termination:
.logout - login shells.
Others:
.history - saves the history (based on $savehist).
tcsh
Start-up (in this order):
/etc/csh.cshrc - always.
/etc/csh.login - login shells.
.tcshrc - always.
.cshrc - if no .tcshrc was present.
.login - login shells
Upon termination:
.logout - login shells.
Others:
.history - saves the history (based on $savehist).
.cshdirs - saves the directory stack.
sh
Start-up (in this order):
/etc/profile - login shells.
.profile - login shells.
Upon termination:
any command (or script) specified using the command:
trap "command" 0
ksh
Start-up (in this order):
/etc/profile - login shells.
.profile - login shells; unless the -p option is used.
$ENV - always, if it is set; unless the -p option is used.
/etc/suid_profile - when the -p option is used.
Upon termination:
any command (or script) specified using the command:
trap "command" 0
bash
Start-up (in this order):
/etc/profile - login shells.
.bash_profile - login shells.
.profile - login if no .bash_profile is present.
.bashrc - interactive non-login shells.
$ENV - always, if it is set.
Upon termination:
.bash_logout - login shells.
Others:
.inputrc - Readline initialization.
zsh
Start-up (in this order):
.zshenv - always, unless -f is specified.
.zprofile - login shells.
.zshrc - interactive shells, unless -f is specified.
.zlogin - login shells.
Upon termination:
.zlogout - login shells.
rc
Start-up:
.rcrc - login shells
The C shell is a command processor that's typically run in a text window, allowing the user to type commands which cause actions. The C shell can also read commands from a file, called a script. Like all Unix shells, it supports filename wildcarding, piping, here documents, command substitution, variables and control structures for condition-testing and iteration. What differentiated the C shell, especially in the 1980s, were its interactive features and overall style. Its new features made it easier and faster to use. The overall style of the language looked more like C and was seen as more readable.
tcsh (TENEX C-Shell)
On many systems such as Mac OS X and Red Hat Linux csh is actually tcsh, an improved version of csh. One file containing the tcsh executable has links to it as both "csh" and "tcsh" so that either name refers to the same improved version of the C shell.
On Ubuntu and Solaris machines, there are two different packages: csh and tcsh. The former is a based on the original BSD version of csh and the later is the improved tcsh. To enable the improved version of csh (tcsh), just type 'tcsh'... You will now notice you have access to command history, working arrows etc...
Other shells exist, such as: bash, sh, ksh, zsh, rc etc... Each having their own unique advantages, however, the common shell principles still apply.
Shell startup Scripts (.files)
When you enter a shell (by logging in for example) or switch to another shell (su or executing a different shell), certain shell startup scripts will be executed. It can become confusing as to which shells use which files (I.e. .cshrc, .profile etc etc).
The following as a list of which shells use which files...
csh
Some versions have system-wide .cshrc and .login files. Every
version puts them in different places.
Start-up (in this order):
.cshrc - always; unless the -f option is used.
.login - login shells.
Upon termination:
.logout - login shells.
Others:
.history - saves the history (based on $savehist).
tcsh
Start-up (in this order):
/etc/csh.cshrc - always.
/etc/csh.login - login shells.
.tcshrc - always.
.cshrc - if no .tcshrc was present.
.login - login shells
Upon termination:
.logout - login shells.
Others:
.history - saves the history (based on $savehist).
.cshdirs - saves the directory stack.
sh
Start-up (in this order):
/etc/profile - login shells.
.profile - login shells.
Upon termination:
any command (or script) specified using the command:
trap "command" 0
ksh
Start-up (in this order):
/etc/profile - login shells.
.profile - login shells; unless the -p option is used.
$ENV - always, if it is set; unless the -p option is used.
/etc/suid_profile - when the -p option is used.
Upon termination:
any command (or script) specified using the command:
trap "command" 0
bash
Start-up (in this order):
/etc/profile - login shells.
.bash_profile - login shells.
.profile - login if no .bash_profile is present.
.bashrc - interactive non-login shells.
$ENV - always, if it is set.
Upon termination:
.bash_logout - login shells.
Others:
.inputrc - Readline initialization.
zsh
Start-up (in this order):
.zshenv - always, unless -f is specified.
.zprofile - login shells.
.zshrc - interactive shells, unless -f is specified.
.zlogin - login shells.
Upon termination:
.zlogout - login shells.
rc
Start-up:
.rcrc - login shells
Labels:
Unix
pkg-get and pkgutil package management for Solaris/SunOS
pkg-get
Pkg-get is an open-source, Sun-approved software installation and management tool for Sun Solaris. You can use pkg-get to install a variety of pre-compiled software without having to worry about version compatibility, patch requirements, libraries, or any other prerequisites.
pkgutil
Pkgutil, written in Perl and licensed under GPL, is a tool to make installation of packages in Solaris easier. It handles package dependencies so all required packages are installed before the desired package automatically. This is based on functionality present in apt-get (Debian Linux).
Which tool?
Both tools offer a similar deal... They aide package management on SunOS, they work out dependencies for packages and versioning is controlled by the package manager. I personally use pkgutil, simply because The Open Community Software Project (OpenCSW) has now deprecated support for pkg-get users (This is a large database of free Solaris packages - Note: There are many others you can use with pkg-get)
OpenCSW/Blastwave
The Open Community Software Project (OpenCSW) is an open-source project providing Solaris binary packages of freely available or open-source software. This was previously hosted at Blastwave. The project was originally Blastwave, but this was forked to created The OpenCSW.
OpenCSW provides packages for Solaris 9 and 10, for 32 and 64-bit, x86 and SPARC architectures. Solaris 8 is no longer a 1st-class supported OS[2]: however, there still exists a legacy Solaris 8 archive, which occasionally gets updates.
This isn't the only place to retrieve Solaris packages, but with a nice large database (3500+ packages at time of writing), its quite beneficial to consume.
Getting Started with pkgutil
http://www.blastwave.org/jir/blastwave.fam
Mirrors
OpenCSW/Blastwave do not directly host packages... A number of mirror sites are available to get these, but you must set this in your pkgutil config file. This is typically located here: /etc/opt/csw/pkgutil.conf
To view a list of mirrors, visit The OpenCSW Mirror Page
When selecting a mirror, you must add '/stable' or '/unstable' to the end of the URL. Unstable packages will be more up to date, but may have bugs present. Stable, on the other hand, will have older packages but more robust.
Open your configuration file for editing and alter the "mirror" section. The below example is a stable UK mirror site.
Save the file and attempt to update the pkgutil catelog (Basically an index of all availalbe packages for a mirror).
Update the pkgutil catelog
Now the catelog is up to date, we can begin to use pkgutil to download and install packages.
Things to note
Ensure wget is in your system PATH (which wget)... pkgutil ships with it's own version, but most distributions of Solaris have a version located here: /usr/sfw/bin
Update pkgutil using the following command
Install packages using the following (I.e. bash)
Pkg-get is an open-source, Sun-approved software installation and management tool for Sun Solaris. You can use pkg-get to install a variety of pre-compiled software without having to worry about version compatibility, patch requirements, libraries, or any other prerequisites.
pkgutil
Pkgutil, written in Perl and licensed under GPL, is a tool to make installation of packages in Solaris easier. It handles package dependencies so all required packages are installed before the desired package automatically. This is based on functionality present in apt-get (Debian Linux).
Which tool?
Both tools offer a similar deal... They aide package management on SunOS, they work out dependencies for packages and versioning is controlled by the package manager. I personally use pkgutil, simply because The Open Community Software Project (OpenCSW) has now deprecated support for pkg-get users (This is a large database of free Solaris packages - Note: There are many others you can use with pkg-get)
OpenCSW/Blastwave
The Open Community Software Project (OpenCSW) is an open-source project providing Solaris binary packages of freely available or open-source software. This was previously hosted at Blastwave. The project was originally Blastwave, but this was forked to created The OpenCSW.
OpenCSW provides packages for Solaris 9 and 10, for 32 and 64-bit, x86 and SPARC architectures. Solaris 8 is no longer a 1st-class supported OS[2]: however, there still exists a legacy Solaris 8 archive, which occasionally gets updates.
This isn't the only place to retrieve Solaris packages, but with a nice large database (3500+ packages at time of writing), its quite beneficial to consume.
Getting Started with pkgutil
http://www.blastwave.org/jir/blastwave.fam
Mirrors
OpenCSW/Blastwave do not directly host packages... A number of mirror sites are available to get these, but you must set this in your pkgutil config file. This is typically located here: /etc/opt/csw/pkgutil.conf
To view a list of mirrors, visit The OpenCSW Mirror Page
When selecting a mirror, you must add '/stable' or '/unstable' to the end of the URL. Unstable packages will be more up to date, but may have bugs present. Stable, on the other hand, will have older packages but more robust.
Open your configuration file for editing and alter the "mirror" section. The below example is a stable UK mirror site.
Code Snippet
- mirror=http://www.grangefields.co.uk/mirrors/csw/stable/
End of Code Snippet
Save the file and attempt to update the pkgutil catelog (Basically an index of all availalbe packages for a mirror).
Update the pkgutil catelog
Code Snippet
- pkgutil -U
End of Code Snippet
Now the catelog is up to date, we can begin to use pkgutil to download and install packages.
Things to note
Ensure wget is in your system PATH (which wget)... pkgutil ships with it's own version, but most distributions of Solaris have a version located here: /usr/sfw/bin
Update pkgutil using the following command
Code Snippet
- pkgutil -u pkgutil
End of Code Snippet
Install packages using the following (I.e. bash)
Code Snippet
- pkgutil -i bash
End of Code Snippet
Labels:
Solaris
Thursday, 22 March 2012
Wednesday, 21 March 2012
Rar Command Line Usage Examples
Rar a single file/directory
Rar multiple files
Unrar a *.rar compressed file
List the contents of a rar file without uncompressing it
Code Snippet
- rar a NameOfYourArchive.rar /path/to/file
End of Code Snippet
Rar multiple files
Code Snippet
- rar a NameOfYourArchive.rar /path/to/files/*
End of Code Snippet
Unrar a *.rar compressed file
Code Snippet
- unrar x NameOfYourArchive.rar
End of Code Snippet
List the contents of a rar file without uncompressing it
Code Snippet
- unrar l NameOfYourArchive.rar
End of Code Snippet
Restarting and managing services on Mac OS X
Method 1
Searching running processes to get he PID, then killing it (The service will automatically restart).
Example
23450 ?? S 0:00.12 /usr/sbin/sshd -i
23455 ?? S 0:00.03 /usr/sbin/sshd -i
23649 p8 S+ 0:00.00 grep -i ssh
Now use 'kill' on the PID...
Method 2
launchd is a unified, open-source service management framework for starting, stopping and managing daemons, applications, processes, and scripts. Written and designed by Dave Zarzycki at Apple, it was introduced with Mac OS X Tiger and is licensed under the Apache License.
We will be using launchctl, a subset of this framework to manage services...
List all services
We can then use this list to identify the names of our services.
Here's an example with SSH...
Stopping a service
Starting a service
Searching running processes to get he PID, then killing it (The service will automatically restart).
Example
Code Snippet
- sudo ps -A | grep -i ssh
End of Code Snippet
23450 ?? S 0:00.12 /usr/sbin/sshd -i
23455 ?? S 0:00.03 /usr/sbin/sshd -i
23649 p8 S+ 0:00.00 grep -i ssh
Now use 'kill' on the PID...
Code Snippet
- sudo kill 23450
End of Code Snippet
Method 2
launchd is a unified, open-source service management framework for starting, stopping and managing daemons, applications, processes, and scripts. Written and designed by Dave Zarzycki at Apple, it was introduced with Mac OS X Tiger and is licensed under the Apache License.
We will be using launchctl, a subset of this framework to manage services...
List all services
Code Snippet
- launchctl list
End of Code Snippet
We can then use this list to identify the names of our services.
Here's an example with SSH...
Stopping a service
Code Snippet
- launchctl stop com.openssh.sshd
End of Code Snippet
Starting a service
Code Snippet
- launchctl start com.openssh.sshd
End of Code Snippet
Labels:
Mac OS X
How to enable X11 Forwarding with SSH on Mac OS X
The aim of this blog post is to discuss how to display an X window from a target machine on our local system.
A good example here would be the Oracle installer for linux... when the installer is executed, it utilises the windowing system configured to display the installer via. SSH. Lets see how we can set this up...
What is the X windowing system (or X11)
The X window system (commonly X Window System or X11, based on its current major version being 11) is a computer software system and network protocol that provides a basis for graphical user interfaces (GUIs) and rich input device capability for networked computers.
Advantages of using SSH for X11 Traffic
- No problems with firewalls, as long as they let SSH pass through (No need to allow access to tcp-port 6000+)
- Communication is encrypted
Note to other OS users [Windows etc]
As long as your software/OS implements the X windowing system, you will be able to display X windows. For windows users, I recommend PuTTY
Mac OS X Configuration
Enable X11 Forwarding in sshd_config
In order to allow X11 forwarding to pass through SSH, we need to enable X11 Forwarding in our config for SSH. This is located in a file called sshd_config and found typically here: /etc/sshd_config or /etc/ssh/sshd_config
Uncomment out the line "# X11Forwarding no" and set it to "yes" (If this hasn't been done already. You will be left with this configuration...
Save the file, and restart the sshd service...
Add your target system to the access control list
So our windowing system on our local machine has an access control list (We cant let anybody in as and when we please!)... So we need to allow our target system access to our windowing system.
You will then see this message to confirm...
fullyqualified.server.name being added to access control list
SSH onto our target machine (with the apps using the windowing system. I.e. Oracle Installer)
We can now SSH onto our target machine...
Ensure the DISPLAY environment variable is configured correctly
On the machine we just SSH'd onto, we need to make sure the DISPLAY environment variable is configured correctly. Our apps will use this to find out where our target machine that supports the X windowing system is located.
Ensure this has been set correctly...
Now lets test to see if our X apps can use our host windowing system via. SSH...
If you do not have xclock, or maybe your trying to execute an installer for Oracle or something, then try and execute the installer.
Gotchas
- Do not set the DISPLAY variable in any login scripts (I.e. bashrc, bash_profile etc...)
A good example here would be the Oracle installer for linux... when the installer is executed, it utilises the windowing system configured to display the installer via. SSH. Lets see how we can set this up...
What is the X windowing system (or X11)
The X window system (commonly X Window System or X11, based on its current major version being 11) is a computer software system and network protocol that provides a basis for graphical user interfaces (GUIs) and rich input device capability for networked computers.
Advantages of using SSH for X11 Traffic
- No problems with firewalls, as long as they let SSH pass through (No need to allow access to tcp-port 6000+)
- Communication is encrypted
Note to other OS users [Windows etc]
As long as your software/OS implements the X windowing system, you will be able to display X windows. For windows users, I recommend PuTTY
Mac OS X Configuration
Enable X11 Forwarding in sshd_config
In order to allow X11 forwarding to pass through SSH, we need to enable X11 Forwarding in our config for SSH. This is located in a file called sshd_config and found typically here: /etc/sshd_config or /etc/ssh/sshd_config
Uncomment out the line "# X11Forwarding no" and set it to "yes" (If this hasn't been done already. You will be left with this configuration...
Code Snippet
- X11Forwarding yes
End of Code Snippet
Save the file, and restart the sshd service...
Add your target system to the access control list
So our windowing system on our local machine has an access control list (We cant let anybody in as and when we please!)... So we need to allow our target system access to our windowing system.
Code Snippet
- xhost fullyqualified.server.name
End of Code Snippet
You will then see this message to confirm...
fullyqualified.server.name being added to access control list
SSH onto our target machine (with the apps using the windowing system. I.e. Oracle Installer)
We can now SSH onto our target machine...
Code Snippet
- ssh -X user@fullyqualified.server.name
End of Code Snippet
Ensure the DISPLAY environment variable is configured correctly
On the machine we just SSH'd onto, we need to make sure the DISPLAY environment variable is configured correctly. Our apps will use this to find out where our target machine that supports the X windowing system is located.
Code Snippet
- DISPLAY=fullyqualifiedaddressforourhostmachine:0.0
End of Code Snippet
Ensure this has been set correctly...
Code Snippet
- echo $DISPLAY
End of Code Snippet
Now lets test to see if our X apps can use our host windowing system via. SSH...
Code Snippet
- xclock &
End of Code Snippet
If you do not have xclock, or maybe your trying to execute an installer for Oracle or something, then try and execute the installer.
Gotchas
- Do not set the DISPLAY variable in any login scripts (I.e. bashrc, bash_profile etc...)
Friday, 9 March 2012
Python - POST HTTP multipart form (Using standard urllib libraries)
The Python script in this example utilises standard Python libraries to POST a form with content type multipart/form-data over HTTP (or quite simply, a multipart form).
The content type "multipart/form-data" should be used for submitting forms that contain files, non-ASCII data, and binary data.
I won't delve deep into the details on multipart forms, but if you would like a good explanation, read the official W3C document. One thing if doesn't mention on there, is that the boundary can be between 1 and 70 characters long, consisting of alphanumeric, and the punctuation you see in the list. Spaces are allowed except at the end.
The content type "multipart/form-data" should be used for submitting forms that contain files, non-ASCII data, and binary data.
I won't delve deep into the details on multipart forms, but if you would like a good explanation, read the official W3C document. One thing if doesn't mention on there, is that the boundary can be between 1 and 70 characters long, consisting of alphanumeric, and the punctuation you see in the list. Spaces are allowed except at the end.
Code Snippet
- import itertools
- import mimetools
- import mimetypes
- from cStringIO import StringIO
- import urllib
- import urllib2
- class MultiPartForm(object):
- """Accumulate the data to be used when posting a form."""
- def __init__(self):
- self.form_fields = []
- self.files = []
- # Generate unique boundary string with the format: hostipaddr.uid.pid.timestamp.random
- self.boundary = mimetools.choose_boundary()
- return
- def get_content_type(self):
- return 'multipart/form-data; boundary=%s' % self.boundary
- def add_field(self, name, value):
- """Add a simple field to the form data."""
- self.form_fields.append((name, value))
- return
- def add_file(self, fieldname, filename, fileHandle, mimetype=None):
- """Add a file to be uploaded."""
- body = fileHandle.read()
- if mimetype is None:
- mimetype = mimetypes.guess_type(filename)[0] or 'application/octet-stream'
- self.files.append((fieldname, filename, mimetype, body))
- return
- def __str__(self):
- """Return a string representing the form data, including attached files."""
- # Build a list of lists, each containing "lines" of the
- # request. Each part is separated by a boundary string.
- # Once the list is built, return a string where each
- # line is separated by '\r\n'.
- parts = []
- part_boundary = '--' + self.boundary
- # Add the form fields
- parts.extend(
- [ part_boundary,
- 'Content-Disposition: form-data; name="%s"' % name,
- '',
- value,
- ]
- for name, value in self.form_fields
- )
- # Add the files to upload
- parts.extend(
- [ part_boundary,
- 'Content-Disposition: file; name="%s"; filename="%s"' % \
- (field_name, filename),
- 'Content-Type: %s' % content_type,
- '',
- body,
- ]
- for field_name, filename, content_type, body in self.files
- )
- # Flatten the list and add closing boundary marker,
- # then return CR+LF separated data
- flattened = list(itertools.chain(*parts))
- flattened.append('--' + self.boundary + '--')
- flattened.append('')
- return '\r\n'.join(flattened)
- if __name__ == '__main__':
- # Create the form with simple fields
- form = MultiPartForm()
- """
- Either specify a source, file or a url...
- The system will take the first one respectively if all three have content.
- This example uses a dummy file called test.sql, with a string of content.
- """
- # No source specified
- form.add_field('source', '')
- # REAL FILE EXAMPLE
- # fileHandle = open ('test.sql', 'r')
- # form.add_file('file', 'test.sql', fileHandle)
- # fileHandle.close()
- # Use dummy file
- form.add_file('file', 'test.sql', fileHandle=StringIO('FILE CONTENTS'))
- # No URL specified
- form.add_field('url', '')
- # Add remaining required parameters
- form.add_field('language', 'php')
- form.add_field('line_numbers', '2')
- form.add_field('word_wrap', 'on')
- form.add_field('tab_width', '8')
- form.add_field('highlight_keywords', 'on')
- form.add_field('default_color', '0000bb')
- form.add_field('keyword_colors[1]', 'DEFAULT')
- form.add_field('keyword_colors[2]', 'DEFAULT')
- form.add_field('keyword_colors[3]', 'DEFAULT')
- form.add_field('keyword_colors[4]', 'DEFAULT')
- form.add_field('comments_color', 'DEFAULT')
- form.add_field('escaped_chars_color', 'DEFAULT')
- form.add_field('brackets_color', 'DEFAULT')
- form.add_field('strings_color', 'DEFAULT')
- form.add_field('numbers_color', 'DEFAULT')
- form.add_field('methods_color', 'DEFAULT')
- # Build the request
- request = urllib2.Request('http://qbnz.com/highlighter/php_highlighter.php')
- # Hi i'm FireFox 5
- request.add_header('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11')
- # We are required to specify the content type and length...
- body = str(form)
- request.add_header('Content-type', form.get_content_type())
- request.add_header('Content-length', len(body))
- request.add_data(body)
- # Take a look at what we are sending... useful to match up against the required request
- print
- print 'OUTGOING DATA:'
- print request.get_data()
- # View servers response
- print
- print 'SERVER RESPONSE:'
- print urllib2.urlopen(request).read()
End of Code Snippet
Labels:
Python
Thursday, 8 March 2012
GeSHi - Generic Programming Code Syntax Highlighter
Code syntax highlighter written in PHP
http://qbnz.com/highlighter/
You can easily interface with blogger/blogspot by using the demo website and copying out the CSS for your chosen language. Each language will have a different CSS stylesheet, so if you have to tweak one language to fit your blogspot template, then you will need to apply this to the others.
Examples
Default code block stylings
Using GeSHi stylings
Line numbers are excluded when copying a code snippet to the clipboard also, which is another good thing!
With BlogSpot/Blogger, if you add a new post and give it a Label with the same name as a chosen language (I.e. Add Python code to a post, and assign a Python Label/Categroy to the post (which is quite common))... Your blog post will appear garbled. This is because Blogger assigns the label to a class and this will take on your CSS styles from GeSHi
Here is a link to a CSS file I have used with most of the languages I talk about in. It support all the default settings, but with word wrap and line numbers both ON. I have prefixed all CSS classes with "sh_" to avoid this problem. Remember to change the class of the main DIV when taking the styles from GeSHi.
Download link for modified CSS
http://qbnz.com/highlighter/
You can easily interface with blogger/blogspot by using the demo website and copying out the CSS for your chosen language. Each language will have a different CSS stylesheet, so if you have to tweak one language to fit your blogspot template, then you will need to apply this to the others.
Examples
Default code block stylings
This is a standard code block
[<code>This is a standard code block</code>]Using GeSHi stylings
Code Snippet
- This is an example code snippet
- # Here is a comment, btw we are using Bash code highlighting
- if [ $THIS -eq 999 ]; then
- eval "lets evaluate something here"
- fi
- # Have fun!
End of snippet
Line numbers are excluded when copying a code snippet to the clipboard also, which is another good thing!
IMPORTANT UPDATE
With BlogSpot/Blogger, if you add a new post and give it a Label with the same name as a chosen language (I.e. Add Python code to a post, and assign a Python Label/Categroy to the post (which is quite common))... Your blog post will appear garbled. This is because Blogger assigns the label to a class and this will take on your CSS styles from GeSHi
Here is a link to a CSS file I have used with most of the languages I talk about in. It support all the default settings, but with word wrap and line numbers both ON. I have prefixed all CSS classes with "sh_" to avoid this problem. Remember to change the class of the main DIV when taking the styles from GeSHi.
Download link for modified CSS
Labels:
Blogging
Unix Shell Script - Get Tomorrows Date
This code snippet will calculate tomorrows date in the format DDMMYYYY. It takes into account correct monthly overlaps and leap years etc...
Code Snippet
- # default output format, change as necessary
- defaultof="%Y%m%d"
- # check for input format, else use default format,
- # refer to 'man date' for help on format
- # script only supports %Y %m %d at the moment
- of=$defaultof
- [ $# -eq 1 ] && of="$1"
- # get today's date
- eval "`date +'y=%Y m=%m d=%d'`"
- #check for max number of days in current month
- days=31
- if [ $m -eq 4 ] || [ $m -eq 6 ] || [ $m -eq 9 ] || [ $m -eq 11 ] ; then
- days=30
- fi
- # check for leap year if feb
- if [ $m -eq 2 ]; then
- days=28
- leap1=`expr $y % 4`
- leap2=`expr $y % 100`
- leap3=`expr $y % 400`
- if [ $leap1 -eq 0 ] ; then
- if [ $leap2 -gt 0 ] || [ $leap3 -eq 0 ] ; then
- days=29
- fi
- fi
- fi
- # increment date
- if [ $d -eq $days ]; then
- d=1
- m=`expr $m + 1`
- if [ $m -eq 13 ]; then
- m=1
- y=`expr $y + 1`
- fi
- else
- d=`expr $d + 1`
- fi
- #Solaris date does not accept -d
- #date -d "$y-$m-$d" +"$of"
- eval "y=`expr $y + 0` m=`expr $m + 0` d=`expr $d + 0`"
- eval "y=`printf "%04d" $y` m=`printf "%02d" $m` d=`printf "%02d" $d`"
- tomorrowsDate="${d}${m}${y}"
End of snippet
Wednesday, 7 March 2012
iptables/Netfilter - Command line overview
iptables/netfilter
Most distributions of linux have an inbuilt firewall, commonly refered to as iptables. In actuality and more accuratly, it is iptables/netfilter. iptables sits in user space where the user can interact and manage rulesets. Netfilter is a kernal module, built into the kernal, that actually does the filtering
iptables is used to set up, maintain, and inspect the tables of IPv4 packet filter rules in the Linux kernel. IPv6 addresses can be managed with ip6tables.
A number of GUIs can be used to manage the process, but often lack the flexibility of the command line. Lets take a look at the process in more detail...
Chains
With a new installation, three pre-defined chains exist (INPUT, OUTPUT and FORWARD)... You can add new chains, remove them and rename them. They simply serve as a container for holding rules. You will refer to the name of your chain when making new rules and managing existing ones.
Let's assume we stuck with the pre-defined chains...
INPUT - All packets destined for the host computer (I.e. keeping people/processes out!)
OUTPUT - All packets originating from the host computer.
FORWARD - All packets neither destined for nor originating from the host computer, but passing through (routed by) the host computer. This chain is used if you are using your computer as a router.
The ordering of rules in a chain does matter. A packet is checked against each rule in turn. Once a packet matches a specific rule (Starting at rule 1), then the action is taken on that packet (ACCEPT, DROP etc..), all other rules below this are ignored. If a packet makes it all the way down the rule chain with no matches, then the default action for that chain is taken. This is referred to as the default policy and may be set to either ACCEPT or DROP the packet.
Let's see a live example...
To display a list of current rules, execute the following command:
This will display the following...
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
ACCEPT icmp -- anywhere anywhere icmp any
ACCEPT esp -- anywhere anywhere
ACCEPT ah -- anywhere anywhere
ACCEPT udp -- anywhere 224.0.0.251 udp dpt:mdns
ACCEPT udp -- anywhere anywhere udp dpt:ipp
ACCEPT tcp -- anywhere anywhere tcp dpt:ipp
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
You can see that the default policy is to REJECT the packet on all protocols, from any source and to any destination.
You can view all of the commands for iptables by executing the following command:
In the examples section below, I have listed some common commands for managing firewall rules. Below each, discusses the parameters used. I have used a range of commands which allow me to explain the majority of the accepted values in detail...
EXAMPLES
List all IP chains with line numbers
- Lists all rules for all chains, also echo's line numbers top the left (This allows us to insert rules at certain positions)
Insert a rule at position 13 (Chain: INPUT, Protocol: TCP, Port 115)
States
- Here we're using the -m switch to load a module (state). The state module is able to examine the state of a packet and determine if it is NEW, ESTABLISHED or RELATED
- NEW refers to incoming packets that are new incoming connections that weren't initiated by the host system.
- ESTABLISHED and RELATED refers to incoming packets that are part of an already established connection or related to and already established connection.
-j ACCEPT
-j (jump) to the target action for packets matching the rule - in this case ACCEPT.
Append a rule to the bottom of a chosen chain (In this case the INPUT chain) for an interface [localhost])
- 'lo' represents localhost (127.0.0.1)... This is generally required as many software applications expect to be able to communicate with the localhost adaptor.
Remove all rules in all chains
Remove all rules in a chain
Remove a rule (Remove a rule at position 13)
Saving Rules (Persisting changing on system reboot)
Firewall changes need to be saved in order to be persisted on system reboot. We can achieve this by using the following command...
Saving firewall rules to /etc/sysconfig/iptables: [ OK ]
This executes the iptables init script, which runs /sbin/iptables-save and writes the current iptables configuration to /etc/sysconfig/iptables. Upon reboot, the iptables init script reapplies the rules saved in /etc/sysconfig/iptables by using the /sbin/iptables-restore command.
References and more info
http://wiki.centos.org/HowTos/Network/IPTables
Most distributions of linux have an inbuilt firewall, commonly refered to as iptables. In actuality and more accuratly, it is iptables/netfilter. iptables sits in user space where the user can interact and manage rulesets. Netfilter is a kernal module, built into the kernal, that actually does the filtering
iptables is used to set up, maintain, and inspect the tables of IPv4 packet filter rules in the Linux kernel. IPv6 addresses can be managed with ip6tables.
A number of GUIs can be used to manage the process, but often lack the flexibility of the command line. Lets take a look at the process in more detail...
Chains
With a new installation, three pre-defined chains exist (INPUT, OUTPUT and FORWARD)... You can add new chains, remove them and rename them. They simply serve as a container for holding rules. You will refer to the name of your chain when making new rules and managing existing ones.
Let's assume we stuck with the pre-defined chains...
INPUT - All packets destined for the host computer (I.e. keeping people/processes out!)
OUTPUT - All packets originating from the host computer.
FORWARD - All packets neither destined for nor originating from the host computer, but passing through (routed by) the host computer. This chain is used if you are using your computer as a router.
The ordering of rules in a chain does matter. A packet is checked against each rule in turn. Once a packet matches a specific rule (Starting at rule 1), then the action is taken on that packet (ACCEPT, DROP etc..), all other rules below this are ignored. If a packet makes it all the way down the rule chain with no matches, then the default action for that chain is taken. This is referred to as the default policy and may be set to either ACCEPT or DROP the packet.
Let's see a live example...
To display a list of current rules, execute the following command:
Code Snippet
- iptables --list
End of Code Snippet
This will display the following...
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
ACCEPT icmp -- anywhere anywhere icmp any
ACCEPT esp -- anywhere anywhere
ACCEPT ah -- anywhere anywhere
ACCEPT udp -- anywhere 224.0.0.251 udp dpt:mdns
ACCEPT udp -- anywhere anywhere udp dpt:ipp
ACCEPT tcp -- anywhere anywhere tcp dpt:ipp
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
Chain FORWARD (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
You can see that the default policy is to REJECT the packet on all protocols, from any source and to any destination.
You can view all of the commands for iptables by executing the following command:
Code Snippet
- iptables --help
End of Code Snippet
In the examples section below, I have listed some common commands for managing firewall rules. Below each, discusses the parameters used. I have used a range of commands which allow me to explain the majority of the accepted values in detail...
EXAMPLES
List all IP chains with line numbers
Code Snippet
- iptables --list --line-numbers
End of Code Snippet
- Lists all rules for all chains, also echo's line numbers top the left (This allows us to insert rules at certain positions)
Insert a rule at position 13 (Chain: INPUT, Protocol: TCP, Port 115)
Code Snippet
- iptables --insert INPUT 13 -m state --state NEW -p tcp --dport 115 -j ACCEPT
End of Code Snippet
States
- Here we're using the -m switch to load a module (state). The state module is able to examine the state of a packet and determine if it is NEW, ESTABLISHED or RELATED
- NEW refers to incoming packets that are new incoming connections that weren't initiated by the host system.
- ESTABLISHED and RELATED refers to incoming packets that are part of an already established connection or related to and already established connection.
-j ACCEPT
-j (jump) to the target action for packets matching the rule - in this case ACCEPT.
Append a rule to the bottom of a chosen chain (In this case the INPUT chain) for an interface [localhost])
Code Snippet
- iptables --append INPUT -i lo -j ACCEPT
End of Code Snippet
- 'lo' represents localhost (127.0.0.1)... This is generally required as many software applications expect to be able to communicate with the localhost adaptor.
Remove all rules in all chains
Code Snippet
- iptables --flush
End of Code Snippet
Remove all rules in a chain
Code Snippet
- iptables --flush [Chain Name]
End of Code Snippet
Remove a rule (Remove a rule at position 13)
Code Snippet
- iptables --delete INPUT 13
End of Code Snippet
Saving Rules (Persisting changing on system reboot)
Firewall changes need to be saved in order to be persisted on system reboot. We can achieve this by using the following command...
Code Snippet
- service iptables save
End of Code Snippet
Saving firewall rules to /etc/sysconfig/iptables: [ OK ]
This executes the iptables init script, which runs /sbin/iptables-save and writes the current iptables configuration to /etc/sysconfig/iptables. Upon reboot, the iptables init script reapplies the rules saved in /etc/sysconfig/iptables by using the /sbin/iptables-restore command.
References and more info
http://wiki.centos.org/HowTos/Network/IPTables
Tuesday, 6 March 2012
tcpdump localhost
Capture packets on localhost from localhost
Code Snippet
- sudo tcpdump -i lo
End of Code Snippet
Monday, 5 March 2012
WSDL (Web Service Description Language) - Complete Overview
The aim of this article is to describe what WSDL is, how we can utilise the technology from a development perspective and dive into technical detail on how the technology fits together and what is possible. This is an initial draft, but I will be updating it casually over the next few weeks. I have an example Math WSDL file provided by Microsoft throughout this article (Full appendix at the bottom of this post).
WSDL 1.1 (Web Service Description Language)
A WSDL file is an XML file which describes its associated web service. The WSDL XML file describes: where the web service is located, the methods available, parameters required and their return types. WSDLs are platform in-specific and can be consumed from any accessible environment, framework and/or language (Mac/Windows/Linux, Web Apps/Services, C#/C++/Python/Perl etc...)
Using the WSDL file, a consumer has enough information to query the underlying web services using one or many of it's associated operations. Most languages/frameworks have built-in or third party libraries available that can generate proxy classes based on the WSDL file (Another layer in front of the WSDL/Service you can query against using your chosen language. I.e. wsdl.exe for .NET or WSDL2Java for Java). This is only an option to make things easier, and is by no means mandatory.
Version History
WSDL 1.0 was introduced in 2000 and was developed through a collaboration by Microsoft, IBM and Ariba. The language was formalized in 2001 as version 1.1.
WSDL 2.0 became a W3C recommendation on June 2007. WSDL 1.2 was renamed to WSDL 2.0 because it has substantial differences from WSDL 1.1. The changes are:
Adding further semantics to the description language
Removal of message constructs
No support for operator overloading
PortTypes renamed to interfaces
Ports renamed to endpoint.
Offers binding to all HTTP Request methods (Not just GET and POST [WSDL 1.1]).. So it enabled better RESTful web services.
Most third-party vendors, however, have not adapted their offerings to support WSDL 2.0. For example, the Web Services Business Process Execution Language (WS-BPEL) uses WSDL 1.1.
Document Structure (One or many of these base four definitions may exist)
Note:
- A WSDL can contain extension elements. For example, it is also possible to group multiple services into one WSDL. However, the core principles behind the technology remain the same.
- It is common for one or many type, mesaage, portType or binding definitions to exist within one WSDL file.
Port Types (Interfaces in 2.0)
The portType section defines all operations available within the web service. It also defines the input, output and fault messages for each operation (A fault tag can only exist with an output tag). The input/output/fault message names are mapped onto message tag names, which describe the names and types of parameters used.
Example
The above example defines a single portType/Interface with four operations (Add, Subtract, Multiply and Divide). These are all Request-response type operations (because they have input and output) that also include faults.
There are four different types of MEPs (Message exchange patterns) that exist when defining operations in WSDL files...
One-way
- Contains only an input message.
Request-response
- Contains an input message, followed by an output message (may contain a fault message)
Solicit-response
- Contains an output message, followed by an input message (may contain a fault message)
Notification
- Contains only an output message.
I will cover messages later in this article, but I have displays the tags here (For the Add operation only) to see how operations are mapped onto messages. AddFaultMessage takes a simple string type, however, complex types (Arrays, Lists, Custom Objects etc..) may exist which require definition in the Types section of the WSDL. Lets have a look at this next...
Types
The types section defines the data types that are used by the service. If the service uses only simple types I.e. Integers, Strings etc... then the types section is not required. WSDL types can also be reused across multiple services. The types section contains one or many schema tags. XML Schema's are the typical choice, however, WSDL 1.1 is not limited to this schema type.
Example
From our previous example, we can see that all four operations map to eight messages (one input, one output, four operations). These 8 messages map on to these eight elements defined in the Types section. Above these elements, you can see two complex types have been defined: MathInput and MathOutput. When contrasting these examples in terms on O-O programming, you can think of a complexType as an Object; and x and y as properties of this object.
Messages (Do not exist in WSDL 2.0 - Operations refer to types directly)
Messages define the input and output of operations. They either map onto an element or a type (Both defined within the Types section). As discussed within the portType section previously, the operation decides which message is used for it's input and output.
Bindings
Bindings provides specific details on how the portType section will actually be transmitted over the wire. The bindings can be made available via. multiple transports: HTTP GET, HTTP POST and/or SOAP.
You can specify multiple bindings for a single portType. I.e. Making an operation available via GET, POST and SOAP.
The following example illustrates a SOAP/HTTP binding for the MathInterface portType (Defined above)...
Example
Binding Info
- The NAME of the binding must be given a unique name. I.e. MathSoapHttpBinding - so it can be referenced from within the WSDL.
- The TYPE of the binding must map onto the name of the portType (Interface WSDL 2.0)
soap:binding Info
- This element indicates that this is a SOAP 1.1 binding.
- The style of service is "document" (See section below on WSDL Styles)
- The transport protocol is HTTP
- The soap:operation element defines the SOAPAction HTTP header value for each operation
- The soap:body element defines how the message parts appear inside of the SOAP Body element (possible values include literal or encoded)
WSDL Styles
A WSDL SOAP binding can be either a Remote Procedure Call (RPC) style binding or a document style binding. A SOAP binding can also have an encoded use or a literal use.
Using document style in SOAP indicates that the body will contain an XML document, and that the message parts specify the XML elements that will be placed there. Using rpc style in SOAP indicates that the body will contain an XML representation of a method call and that the message parts represent the parameters to the method.
This gives you four style/use models:
RPC/encoded
RPC/literal
Document/encoded
Document/literal
The use attribute specifies the encoding that should be used to translate the abstract message parts into a concrete representation. In the case of 'encoded', the abstract definitions are translated into a concrete format by applying the SOAP encoding rules. In the case of 'literal', the abstract type definitions become the concrete definitions themselves (they're 'literal' definitions). In this case, you can simply inspect the XML Schema type definitions to determine the concrete message format. For example, the Add operation for the above document/literal binding looks like this on the wire:
Using literal definitions is much cleaner and easier for the tools to get right. Using the encoding rules has led to significant interoperability problems across toolkits. It also leads to weird situations like not being able to validate the wire-level message against the original schema definition (since it's abstract and not a true representation of the message)
More info on WSDL Styles here: http://www.ibm.com/developerworks/webservices/library/ws-whichwsdl/
Service
The service element defines a collection of endpoints/ports that expose our bindings (Listed above). Each port must be given a name and assigned a binding.
Example
Full WSDL Example (For all above examples)
References
http://msdn.microsoft.com/en-us/library/ms996486.aspx
TODO: Add more info on WSDL 2.0
WSDL 1.1 (Web Service Description Language)
A WSDL file is an XML file which describes its associated web service. The WSDL XML file describes: where the web service is located, the methods available, parameters required and their return types. WSDLs are platform in-specific and can be consumed from any accessible environment, framework and/or language (Mac/Windows/Linux, Web Apps/Services, C#/C++/Python/Perl etc...)
Using the WSDL file, a consumer has enough information to query the underlying web services using one or many of it's associated operations. Most languages/frameworks have built-in or third party libraries available that can generate proxy classes based on the WSDL file (Another layer in front of the WSDL/Service you can query against using your chosen language. I.e. wsdl.exe for .NET or WSDL2Java for Java). This is only an option to make things easier, and is by no means mandatory.
Version History
WSDL 1.0 was introduced in 2000 and was developed through a collaboration by Microsoft, IBM and Ariba. The language was formalized in 2001 as version 1.1.
WSDL 2.0 became a W3C recommendation on June 2007. WSDL 1.2 was renamed to WSDL 2.0 because it has substantial differences from WSDL 1.1. The changes are:
Adding further semantics to the description language
Removal of message constructs
No support for operator overloading
PortTypes renamed to interfaces
Ports renamed to endpoint.
Offers binding to all HTTP Request methods (Not just GET and POST [WSDL 1.1]).. So it enabled better RESTful web services.
Most third-party vendors, however, have not adapted their offerings to support WSDL 2.0. For example, the Web Services Business Process Execution Language (WS-BPEL) uses WSDL 1.1.
Document Structure (One or many of these base four definitions may exist)
Code Snippet
- <definitions>
- <types>
- The data types used by the web service
- </types>
- <message>
- The messages used by the web service
- </message>
- <portType>
- The operations performed by the web service
- </portType>
- <binding>
- The communication protocols used by the web service
- </binding>
- <service>
- Information regarding the service endpoints.
- </service>
- </definitions>
End of Code Snippet
Note:
- A WSDL can contain extension elements. For example, it is also possible to group multiple services into one WSDL. However, the core principles behind the technology remain the same.
- It is common for one or many type, mesaage, portType or binding definitions to exist within one WSDL file.
Port Types (Interfaces in 2.0)
The portType section defines all operations available within the web service. It also defines the input, output and fault messages for each operation (A fault tag can only exist with an output tag). The input/output/fault message names are mapped onto message tag names, which describe the names and types of parameters used.
Example
Code Snippet
- <message name="AddMessage">
- <part name="parameter" element="ns:Add"/>
- </message>
- <message name="AddResponseMessage">
- <part name="parameter" element="ns:AddResponse"/>
- </message>
- <message name="AddFaultMessage">
- <part name="parameter" type="ns:String"/>
- </message>
- <portType name="MathInterface">
- <operation name="Add">
- <input message="y:AddMessage"/>
- <output message="y:AddResponseMessage"/>
- <fault message="y:AddFaultMessage"/>
- </operation>
- <operation name="Subtract">
- <input message="y:SubtractMessage"/>
- <output message="y:SubtractResponseMessage"/>
- <fault message="y:SubtractFaultMessage"/>
- </operation>
- <operation name="Multiply">
- <input message="y:MultiplyMessage"/>
- <output message="y:MultiplyResponseMessage"/>
- <fault message="y:MultiplyFaultMessage"/>
- </operation>
- <operation name="Divide">
- <input message="y:DivideMessage"/>
- <output message="y:DivideResponseMessage"/>
- <fault message="y:DivideFaultMessage"/>
- </operation>
- </portType>
End of Code Snippet
The above example defines a single portType/Interface with four operations (Add, Subtract, Multiply and Divide). These are all Request-response type operations (because they have input and output) that also include faults.
There are four different types of MEPs (Message exchange patterns) that exist when defining operations in WSDL files...
One-way
- Contains only an input message.
Request-response
- Contains an input message, followed by an output message (may contain a fault message)
Solicit-response
- Contains an output message, followed by an input message (may contain a fault message)
Notification
- Contains only an output message.
I will cover messages later in this article, but I have displays the tags here (For the Add operation only) to see how operations are mapped onto messages. AddFaultMessage takes a simple string type, however, complex types (Arrays, Lists, Custom Objects etc..) may exist which require definition in the Types section of the WSDL. Lets have a look at this next...
Types
The types section defines the data types that are used by the service. If the service uses only simple types I.e. Integers, Strings etc... then the types section is not required. WSDL types can also be reused across multiple services. The types section contains one or many schema tags. XML Schema's are the typical choice, however, WSDL 1.1 is not limited to this schema type.
Example
Code Snippet
- <types>
- <schema targetNamespace="http://example.com/weather.xsd" xmlns="http://www.w3.org/2000/10/XMLSchema">
- <xs:complexType name="MathInput">
- <xs:sequence>
- <xs:element name="x" type="xs:double"/>
- <xs:element name="y" type="xs:double"/>
- </xs:sequence>
- </xs:complexType>
- <xs:complexType name="MathOutput">
- <xs:sequence>
- <xs:element name="result" type="xs:double"/>
- </xs:sequence>
- </xs:complexType>
- <xs:element name="Add" type="MathInput"/>
- <xs:element name="AddResponse" type="MathOutput"/>
- <xs:element name="Subtract" type="MathInput"/>
- <xs:element name="SubtractResponse" type="MathOutput"/>
- <xs:element name="Multiply" type="MathInput"/>
- <xs:element name="MultiplyResponse" type="MathOutput"/>
- <xs:element name="Divide" type="MathInput"/>
- <xs:element name="DivideResponse" type="MathOutput"/>
- </xs:schema>
- </types>
End of Code Snippet
From our previous example, we can see that all four operations map to eight messages (one input, one output, four operations). These 8 messages map on to these eight elements defined in the Types section. Above these elements, you can see two complex types have been defined: MathInput and MathOutput. When contrasting these examples in terms on O-O programming, you can think of a complexType as an Object; and x and y as properties of this object.
Messages (Do not exist in WSDL 2.0 - Operations refer to types directly)
Messages define the input and output of operations. They either map onto an element or a type (Both defined within the Types section). As discussed within the portType section previously, the operation decides which message is used for it's input and output.
Code Snippet
- <message name="AddMessage">
- <part name="parameter" element="ns:Add"/>
- </message>
- <message name="AddResponseMessage">
- <part name="parameter" element="ns:AddResponse"/>
- </message>
- <message name="SubtractMessage">
- <part name="parameter" element="ns:Subtract"/>
- </message>
- <message name="SubtractResponseMessage">
- <part name="parameter" element="ns:SubtractResponse"/>
- </message>
End of Code Snippet
Bindings
Bindings provides specific details on how the portType section will actually be transmitted over the wire. The bindings can be made available via. multiple transports: HTTP GET, HTTP POST and/or SOAP.
You can specify multiple bindings for a single portType. I.e. Making an operation available via GET, POST and SOAP.
The following example illustrates a SOAP/HTTP binding for the MathInterface portType (Defined above)...
Example
Code Snippet
- <definitions
- xmlns="http://schemas.xmlsoap.org/wsdl/"
- xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
- xmlns:xs="http://www.w3.org/2001/XMLSchema"
- xmlns:y="http://example.org/math/"
- xmlns:ns="http://example.org/math/types/"
- targetNamespace="http://example.org/math/">
- ...
- <binding name="MathSoapHttpBinding" type="y:MathInterface">
- <soap:binding style="document"
- transport="http://schemas.xmlsoap.org/soap/http"/>
- <operation name="Add">
- <soap:operation soapAction="http://example.org/math/#Add"/>
- <input>
- <soap:body use="literal"/>
- </input>
- <output>
- <soap:body use="literal"/>
- </output>
- <fault>
- <soap:body use="literal"/>
- </fault>
- </operation>
- ...
- </binding>
- ...
- </definitions>
End of Code Snippet
Binding Info
- The NAME of the binding must be given a unique name. I.e. MathSoapHttpBinding - so it can be referenced from within the WSDL.
- The TYPE of the binding must map onto the name of the portType (Interface WSDL 2.0)
soap:binding Info
- This element indicates that this is a SOAP 1.1 binding.
- The style of service is "document" (See section below on WSDL Styles)
- The transport protocol is HTTP
- The soap:operation element defines the SOAPAction HTTP header value for each operation
- The soap:body element defines how the message parts appear inside of the SOAP Body element (possible values include literal or encoded)
WSDL Styles
A WSDL SOAP binding can be either a Remote Procedure Call (RPC) style binding or a document style binding. A SOAP binding can also have an encoded use or a literal use.
Using document style in SOAP indicates that the body will contain an XML document, and that the message parts specify the XML elements that will be placed there. Using rpc style in SOAP indicates that the body will contain an XML representation of a method call and that the message parts represent the parameters to the method.
This gives you four style/use models:
RPC/encoded
RPC/literal
Document/encoded
Document/literal
The use attribute specifies the encoding that should be used to translate the abstract message parts into a concrete representation. In the case of 'encoded', the abstract definitions are translated into a concrete format by applying the SOAP encoding rules. In the case of 'literal', the abstract type definitions become the concrete definitions themselves (they're 'literal' definitions). In this case, you can simply inspect the XML Schema type definitions to determine the concrete message format. For example, the Add operation for the above document/literal binding looks like this on the wire:
Using literal definitions is much cleaner and easier for the tools to get right. Using the encoding rules has led to significant interoperability problems across toolkits. It also leads to weird situations like not being able to validate the wire-level message against the original schema definition (since it's abstract and not a true representation of the message)
More info on WSDL Styles here: http://www.ibm.com/developerworks/webservices/library/ws-whichwsdl/
Service
The service element defines a collection of endpoints/ports that expose our bindings (Listed above). Each port must be given a name and assigned a binding.
Example
Code Snippet
- <definitions
- xmlns="http://schemas.xmlsoap.org/wsdl/"
- xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
- xmlns:xs="http://www.w3.org/2001/XMLSchema"
- xmlns:y="http://example.org/math/"
- xmlns:ns="http://example.org/math/types/"
- targetNamespace="http://example.org/math/">
- ...
- <service name="MathService">
- <port name="MathEndpoint" binding="y:MathSoapHttpBinding">
- <soap:address
- location="http://localhost/math/math.asmx"/>
- </port>
- </service>
- </definitions>
End of Code Snippet
Full WSDL Example (For all above examples)
Code Snippet
- <definitions xmlns="http://schemas.xmlsoap.org/wsdl/"
- xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
- xmlns:http="http://schemas.xmlsoap.org/wsdl/http/"
- xmlns:xs="http://www.w3.org/2001/XMLSchema"
- xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
- xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/"
- xmlns:y="http://example.org/math/"
- xmlns:ns="http://example.org/math/types/"
- targetNamespace="http://example.org/math/">
- <types>
- <xs:schema targetNamespace="http://example.org/math/types/"
- xmlns="http://example.org/math/types/"
- elementFormDefault="unqualified" attributeFormDefault="unqualified">
- <xs:complexType name="MathInput">
- <xs:sequence>
- <xs:element name="x" type="xs:double"/>
- <xs:element name="y" type="xs:double"/>
- </xs:sequence>
- </xs:complexType>
- <xs:complexType name="MathOutput">
- <xs:sequence>
- <xs:element name="result" type="xs:double"/>
- </xs:sequence>
- </xs:complexType>
- <xs:element name="Add" type="MathInput"/>
- <xs:element name="AddResponse" type="MathOutput"/>
- <xs:element name="Subtract" type="MathInput"/>
- <xs:element name="SubtractResponse" type="MathOutput"/>
- <xs:element name="Multiply" type="MathInput"/>
- <xs:element name="MultiplyResponse" type="MathOutput"/>
- <xs:element name="Divide" type="MathInput"/>
- <xs:element name="DivideResponse" type="MathOutput"/>
- </xs:schema>
- </types>
- <message name="AddMessage">
- <part name="parameters" element="ns:Add"/>
- </message>
- <message name="AddResponseMessage">
- <part name="parameters" element="ns:AddResponse"/>
- </message>
- <message name="SubtractMessage">
- <part name="parameters" element="ns:Subtract"/>
- </message>
- <message name="SubtractResponseMessage">
- <part name="parameters" element="ns:SubtractResponse"/>
- </message>
- <message name="MultiplyMessage">
- <part name="parameters" element="ns:Multiply"/>
- </message>
- <message name="MultiplyResponseMessage">
- <part name="parameters" element="ns:MultiplyResponse"/>
- </message>
- <message name="DivideMessage">
- <part name="parameters" element="ns:Divide"/>
- </message>
- <message name="DivideResponseMessage">
- <part name="parameters" element="ns:DivideResponse"/>
- </message>
- <portType name="MathInterface">
- <operation name="Add">
- <input message="y:AddMessage"/>
- <output message="y:AddResponseMessage"/>
- </operation>
- <operation name="Subtract">
- <input message="y:SubtractMessage"/>
- <output message="y:SubtractResponseMessage"/>
- </operation>
- <operation name="Multiply">
- <input message="y:MultiplyMessage"/>
- <output message="y:MultiplyResponseMessage"/>
- </operation>
- <operation name="Divide">
- <input message="y:DivideMessage"/>
- <output message="y:DivideResponseMessage"/>
- </operation>
- </portType>
- <binding name="MathSoapHttpBinding" type="y:MathInterface">
- <soap:binding style="document"
- transport="http://schemas.xmlsoap.org/soap/http"/>
- <operation name="Add">
- <soap:operation soapAction="http://example.org/math/#Add"/>
- <input>
- <soap:body use="literal"/>
- </input>
- <output>
- <soap:body use="literal"/>
- </output>
- </operation>
- <operation name="Subtract">
- <soap:operation soapAction="http://example.org/math/#Subtract"/>
- <input>
- <soap:body use="literal"/>
- </input>
- <output>
- <soap:body use="literal"/>
- </output>
- </operation>
- <operation name="Multiply">
- <soap:operation soapAction="http://example.org/math/#Multiply"/>
- <input>
- <soap:body use="literal"/>
- </input>
- <output>
- <soap:body use="literal"/>
- </output>
- </operation>
- <operation name="Divide">
- <soap:operation soapAction="http://example.org/math/#Divide"/>
- <input>
- <soap:body use="literal"/>
- </input>
- <output>
- <soap:body use="literal"/>
- </output>
- </operation>
- </binding>
- <service name="MathService">
- <port name="MathEndpoint" binding="y:MathSoapHttpBinding">
- <soap:address location="http://localhost/math/math.asmx"/>
- </port>
- </service>
- </definitions>
End of Code Snippet
References
http://msdn.microsoft.com/en-us/library/ms996486.aspx
TODO: Add more info on WSDL 2.0
Labels:
SOAP,
Web Services,
WSDL
Subscribe to:
Posts (Atom)