Python - POST HTTP multipart form (Using standard urllib libraries)

The Python script in this example utilises standard Python libraries to POST a form with content type multipart/form-data over HTTP (or quite simply, a multipart form).

The content type "multipart/form-data" should be used for submitting forms that contain files, non-ASCII data, and binary data.

I won't delve deep into the details on multipart forms, but if you would like a good explanation, read the official W3C document. One thing if doesn't mention on there, is that the boundary can be between 1 and 70 characters long, consisting of alphanumeric, and the punctuation you see in the list. Spaces are allowed except at the end.

Code Snippet

1.     import itertools
2.     import mimetools
3.     import mimetypes
4.     from cStringIO import StringIO
5.     import urllib
6.     import urllib2
7.      
8.     class MultiPartForm(object):
9.         """Accumulate the data to be used when posting a form."""
10.      
11.         def __init__(self):
12.             self.form_fields = []
13.             self.files = []
14.            
15.             # Generate unique boundary string with the format: hostipaddr.uid.pid.timestamp.random
16.             self.boundary = mimetools.choose_boundary()
17.             return
18.        
19.         def get_content_type(self):
20.             return 'multipart/form-data; boundary=%s' % self.boundary
21.      
22.         def add_field(self, name, value):
23.             """Add a simple field to the form data."""
24.             self.form_fields.append((name, value))
25.             return
26.      
27.         def add_file(self, fieldname, filename, fileHandle, mimetype=None):
28.             """Add a file to be uploaded."""
29.             body = fileHandle.read()
30.             if mimetype is None:
31.                 mimetype = mimetypes.guess_type(filename)[0] or 'application/octet-stream'
32.             self.files.append((fieldname, filename, mimetype, body))
33.             return
34.        
35.         def __str__(self):
36.             """Return a string representing the form data, including attached files."""
37.             # Build a list of lists, each containing "lines" of the
38.             # request.  Each part is separated by a boundary string.
39.             # Once the list is built, return a string where each
40.             # line is separated by '\r\n'.  
41.             parts = []
42.             part_boundary = '--' + self.boundary
43.            
44.             # Add the form fields
45.             parts.extend(
46.                 [ part_boundary,
47.                   'Content-Disposition: form-data; name="%s"' % name,
48.                   '',
49.                   value,
50.                 ]
51.                 for name, value in self.form_fields
52.                 )
53.            
54.             # Add the files to upload
55.             parts.extend(
56.                 [ part_boundary,
57.                   'Content-Disposition: file; name="%s"; filename="%s"' % \
58.                      (field_name, filename),
59.                   'Content-Type: %s' % content_type,
60.                   '',
61.                   body,
62.                 ]
63.                 for field_name, filename, content_type, body in self.files
64.                 )
65.            
66.             # Flatten the list and add closing boundary marker,
67.             # then return CR+LF separated data
68.             flattened = list(itertools.chain(*parts))
69.             flattened.append('--' + self.boundary + '--')
70.             flattened.append('')
71.             return '\r\n'.join(flattened)
72.      
73.      
74.      
75.      
76.      
77.     if __name__ == '__main__':
78.      
79.         # Create the form with simple fields
80.         form = MultiPartForm()
81.        
82.        
83.         """
84.            Either specify a source, file or a url...
85.            The system will take the first one respectively if all three have content.
86.            This example uses a dummy file called test.sql, with a string of content.
87.       """
88.        
89.         # No source specified
90.         form.add_field('source', '')
91.        
92.         # REAL FILE EXAMPLE
93.         # fileHandle = open ('test.sql', 'r')
94.         # form.add_file('file', 'test.sql', fileHandle)
95.         # fileHandle.close()
96.        
97.         # Use dummy file
98.         form.add_file('file', 'test.sql', fileHandle=StringIO('FILE CONTENTS'))
99.        
100.         # No URL specified
101.         form.add_field('url', '')
102.        
103.        
104.        
105.         # Add remaining required parameters
106.         form.add_field('language', 'php')
107.         form.add_field('line_numbers', '2')
108.         form.add_field('word_wrap', 'on')
109.         form.add_field('tab_width', '8')
110.         form.add_field('highlight_keywords', 'on')
111.         form.add_field('default_color', '0000bb')
112.         form.add_field('keyword_colors[1]', 'DEFAULT')
113.         form.add_field('keyword_colors[2]', 'DEFAULT')
114.         form.add_field('keyword_colors[3]', 'DEFAULT')
115.         form.add_field('keyword_colors[4]', 'DEFAULT')
116.         form.add_field('comments_color', 'DEFAULT')
117.         form.add_field('escaped_chars_color', 'DEFAULT')
118.         form.add_field('brackets_color', 'DEFAULT')
119.         form.add_field('strings_color', 'DEFAULT')
120.         form.add_field('numbers_color', 'DEFAULT')
121.         form.add_field('methods_color', 'DEFAULT')
122.      
123.         # Build the request
124.         request = urllib2.Request('http://qbnz.com/highlighter/php_highlighter.php')
125.        
126.         # Hi i'm FireFox 5
127.         request.add_header('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11')
128.        
129.         # We are required to specify the content type and length...
130.         body = str(form)
131.         request.add_header('Content-type', form.get_content_type())
132.         request.add_header('Content-length', len(body))
133.         request.add_data(body)
134.      
135.             # Take a look at what we are sending... useful to match up against the required request
136.         print
137.         print 'OUTGOING DATA:'
138.         print request.get_data()
139.      
140.             # View servers response
141.         print
142.         print 'SERVER RESPONSE:'
143.         print urllib2.urlopen(request).read()

End of Code Snippet

GeSHi - Generic Programming Code Syntax Highlighter

Code syntax highlighter written in PHP
http://qbnz.com/highlighter/

You can easily interface with blogger/blogspot by using the demo website and copying out the CSS for your chosen language. Each language will have a different CSS stylesheet, so if you have to tweak one language to fit your blogspot template, then you will need to apply this to the others.

Examples

Default code block stylings
This is a standard code block [<code>This is a standard code block</code>]


Using GeSHi stylings

Code Snippet

1. This is an example code snippet
2. # Here is a comment, btw we are using Bash code highlighting
3.  
4.   if [ $THIS -eq 999 ]; then
5.     eval "lets evaluate something here"
6.   fi
7.  
8. # Have fun!

End of snippet

Line numbers are excluded when copying a code snippet to the clipboard also, which is another good thing!

IMPORTANT UPDATE

With BlogSpot/Blogger, if you add a new post and give it a Label with the same name as a chosen language (I.e. Add Python code to a post, and assign a Python Label/Categroy to the post (which is quite common))... Your blog post will appear garbled. This is because Blogger assigns the label to a class and this will take on your CSS styles from GeSHi

Here is a link to a CSS file I have used with most of the languages I talk about in. It support all the default settings, but with word wrap and line numbers both ON. I have prefixed all CSS classes with "sh_" to avoid this problem. Remember to change the class of the main DIV when taking the styles from GeSHi.

Download link for modified CSS

Unix Shell Script - Get Tomorrows Date

This code snippet will calculate tomorrows date in the format DDMMYYYY. It takes into account correct monthly overlaps and leap years etc...

Code Snippet

1.   # default output format, change as necessary
2.   defaultof="%Y%m%d"
3.  
4.   # check for input format, else use default format,
5.   # refer to 'man date' for help on format
6.   # script only supports %Y %m %d at the moment
7.   of=$defaultof
8.   [ $# -eq 1 ] && of="$1"
9.  
10.   # get today's date
11.   eval "`date +'y=%Y m=%m d=%d'`"
12.  
13.   #check for max number of days in current month
14.   days=31
15.   if [ $m -eq 4 ] || [ $m -eq 6 ] || [ $m -eq 9 ] || [ $m -eq 11 ] ; then
16.     days=30
17.   fi
18.   # check for leap year if feb
19.   if [ $m -eq 2 ]; then
20.     days=28
21.     leap1=`expr $y % 4`
22.     leap2=`expr $y % 100`
23.     leap3=`expr $y % 400`
24.     if [ $leap1 -eq 0 ] ; then
25.       if [ $leap2 -gt 0 ] || [ $leap3 -eq 0 ] ; then
26.         days=29
27.       fi
28.     fi
29.   fi
30.  
31.   # increment date
32.   if [ $d -eq $days ]; then
33.     d=1
34.     m=`expr $m + 1`
35.     if [ $m -eq 13 ]; then
36.       m=1
37.       y=`expr $y + 1`
38.     fi
39.   else
40.     d=`expr $d + 1`
41.   fi
42.  
43.   #Solaris date does not accept -d
44.   #date -d "$y-$m-$d" +"$of"
45.  
46.   eval "y=`expr $y + 0` m=`expr $m + 0` d=`expr $d + 0`"
47.   eval "y=`printf "%04d" $y` m=`printf "%02d" $m` d=`printf "%02d" $d`"
48.   tomorrowsDate="${d}${m}${y}"

End of snippet

iptables/Netfilter - Command line overview

iptables/netfilter
Most distributions of linux have an inbuilt firewall, commonly refered to as iptables. In actuality and more accuratly, it is iptables/netfilter. iptables sits in user space where the user can interact and manage rulesets. Netfilter is a kernal module, built into the kernal, that actually does the filtering

iptables is used to set up, maintain, and inspect the tables of IPv4 packet filter rules in the Linux kernel. IPv6 addresses can be managed with ip6tables.

A number of GUIs can be used to manage the process, but often lack the flexibility of the command line. Lets take a look at the process in more detail...

Chains
With a new installation, three pre-defined chains exist (INPUT, OUTPUT and FORWARD)... You can add new chains, remove them and rename them. They simply serve as a container for holding rules. You will refer to the name of your chain when making new rules and managing existing ones.

Let's assume we stuck with the pre-defined chains...

INPUT - All packets destined for the host computer (I.e. keeping people/processes out!)
OUTPUT - All packets originating from the host computer.
FORWARD - All packets neither destined for nor originating from the host computer, but passing through (routed by) the host computer. This chain is used if you are using your computer as a router.


The ordering of rules in a chain does matter. A packet is checked against each rule in turn. Once a packet matches a specific rule (Starting at rule 1), then the action is taken on that packet (ACCEPT, DROP etc..), all other rules below this are ignored. If a packet makes it all the way down the rule chain with no matches, then the default action for that chain is taken. This is referred to as the default policy and may be set to either ACCEPT or DROP the packet.


Let's see a live example...


To display a list of current rules, execute the following command:

Code Snippet

1. iptables --list

End of Code Snippet

This will display the following...

Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
ACCEPT icmp -- anywhere anywhere icmp any
ACCEPT esp -- anywhere anywhere
ACCEPT ah -- anywhere anywhere
ACCEPT udp -- anywhere 224.0.0.251 udp dpt:mdns
ACCEPT udp -- anywhere anywhere udp dpt:ipp
ACCEPT tcp -- anywhere anywhere tcp dpt:ipp
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited


Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination


You can see that the default policy is to REJECT the packet on all protocols, from any source and to any destination.

You can view all of the commands for iptables by executing the following command:

Code Snippet

1. iptables --help

End of Code Snippet

In the examples section below, I have listed some common commands for managing firewall rules. Below each, discusses the parameters used. I have used a range of commands which allow me to explain the majority of the accepted values in detail...


EXAMPLES


List all IP chains with line numbers

Code Snippet

1. iptables --list --line-numbers

End of Code Snippet

- Lists all rules for all chains, also echo's line numbers top the left (This allows us to insert rules at certain positions)

Insert a rule at position 13 (Chain: INPUT, Protocol: TCP, Port 115)

Code Snippet

1. iptables --insert INPUT 13 -m state --state NEW -p tcp --dport 115 -j ACCEPT

End of Code Snippet

States
- Here we're using the -m switch to load a module (state). The state module is able to examine the state of a packet and determine if it is NEW, ESTABLISHED or RELATED
- NEW refers to incoming packets that are new incoming connections that weren't initiated by the host system.
- ESTABLISHED and RELATED refers to incoming packets that are part of an already established connection or related to and already established connection.

-j ACCEPT
-j (jump) to the target action for packets matching the rule - in this case ACCEPT.


Append a rule to the bottom of a chosen chain (In this case the INPUT chain) for an interface [localhost])

Code Snippet

1. iptables --append INPUT -i lo -j ACCEPT

End of Code Snippet

- 'lo' represents localhost (127.0.0.1)... This is generally required as many software applications expect to be able to communicate with the localhost adaptor.

Remove all rules in all chains

Code Snippet

1. iptables --flush

End of Code Snippet

Remove all rules in a chain

Code Snippet

1. iptables --flush [Chain Name]

End of Code Snippet

Remove a rule (Remove a rule at position 13)

Code Snippet

1. iptables --delete INPUT 13

End of Code Snippet

Saving Rules (Persisting changing on system reboot)

Firewall changes need to be saved in order to be persisted on system reboot. We can achieve this by using the following command...

Code Snippet

1. service iptables save

End of Code Snippet

Saving firewall rules to /etc/sysconfig/iptables: [ OK ]


This executes the iptables init script, which runs /sbin/iptables-save and writes the current iptables configuration to /etc/sysconfig/iptables. Upon reboot, the iptables init script reapplies the rules saved in /etc/sysconfig/iptables by using the /sbin/iptables-restore command.



References and more info
http://wiki.centos.org/HowTos/Network/IPTables

tcpdump localhost

Capture packets on localhost from localhost

Code Snippet

1. sudo tcpdump -i lo

End of Code Snippet

WSDL (Web Service Description Language) - Complete Overview

The aim of this article is to describe what WSDL is, how we can utilise the technology from a development perspective and dive into technical detail on how the technology fits together and what is possible. This is an initial draft, but I will be updating it casually over the next few weeks. I have an example Math WSDL file provided by Microsoft throughout this article (Full appendix at the bottom of this post).


WSDL 1.1 (Web Service Description Language)

A WSDL file is an XML file which describes its associated web service. The WSDL XML file describes: where the web service is located, the methods available, parameters required and their return types. WSDLs are platform in-specific and can be consumed from any accessible environment, framework and/or language (Mac/Windows/Linux, Web Apps/Services, C#/C++/Python/Perl etc...)

Using the WSDL file, a consumer has enough information to query the underlying web services using one or many of it's associated operations. Most languages/frameworks have built-in or third party libraries available that can generate proxy classes based on the WSDL file (Another layer in front of the WSDL/Service you can query against using your chosen language. I.e. wsdl.exe for .NET or WSDL2Java for Java). This is only an option to make things easier, and is by no means mandatory.



Version History

WSDL 1.0 was introduced in 2000 and was developed through a collaboration by Microsoft, IBM and Ariba. The language was formalized in 2001 as version 1.1.

WSDL 2.0 became a W3C recommendation on June 2007. WSDL 1.2 was renamed to WSDL 2.0 because it has substantial differences from WSDL 1.1. The changes are:

Adding further semantics to the description language
Removal of message constructs
No support for operator overloading
PortTypes renamed to interfaces
Ports renamed to endpoint.
Offers binding to all HTTP Request methods (Not just GET and POST [WSDL 1.1]).. So it enabled better RESTful web services.

Most third-party vendors, however, have not adapted their offerings to support WSDL 2.0. For example, the Web Services Business Process Execution Language (WS-BPEL) uses WSDL 1.1.




Document Structure (One or many of these base four definitions may exist)

Code Snippet

1. <definitions>
2.  
3. <types>
4.   The data types used by the web service
5. </types>
6.  
7. <message>
8.   The messages used by the web service
9. </message>
10.  
11. <portType>
12.   The operations performed by the web service
13. </portType>
14.  
15. <binding>
16.   The communication protocols used by the web service
17. </binding>
18.  
19. <service>
20.   Information regarding the service endpoints.
21. </service>
22.  
23. </definitions>

End of Code Snippet

Note:
- A WSDL can contain extension elements. For example, it is also possible to group multiple services into one WSDL. However, the core principles behind the technology remain the same.
- It is common for one or many type, mesaage, portType or binding definitions to exist within one WSDL file.




Port Types (Interfaces in 2.0)

The portType section defines all operations available within the web service. It also defines the input, output and fault messages for each operation (A fault tag can only exist with an output tag). The input/output/fault message names are mapped onto message tag names, which describe the names and types of parameters used.

Example

Code Snippet

1. <message name="AddMessage">
2. <part name="parameter" element="ns:Add"/>
3. </message>
4. <message name="AddResponseMessage">
5. <part name="parameter" element="ns:AddResponse"/>
6. </message>
7. <message name="AddFaultMessage">
8. <part name="parameter" type="ns:String"/>
9. </message>
10.  
11.  
12. <portType name="MathInterface">
13. <operation name="Add">
14. <input message="y:AddMessage"/>
15. <output message="y:AddResponseMessage"/>
16. <fault message="y:AddFaultMessage"/>
17. </operation>
18. <operation name="Subtract">
19. <input message="y:SubtractMessage"/>
20. <output message="y:SubtractResponseMessage"/>
21. <fault  message="y:SubtractFaultMessage"/>
22. </operation>
23. <operation name="Multiply">
24. <input message="y:MultiplyMessage"/>
25. <output message="y:MultiplyResponseMessage"/>
26. <fault  message="y:MultiplyFaultMessage"/>
27. </operation>
28. <operation name="Divide">
29. <input message="y:DivideMessage"/>
30. <output message="y:DivideResponseMessage"/>
31. <fault  message="y:DivideFaultMessage"/>
32. </operation>
33. </portType>

End of Code Snippet

The above example defines a single portType/Interface with four operations (Add, Subtract, Multiply and Divide). These are all Request-response type operations (because they have input and output) that also include faults.

There are four different types of MEPs (Message exchange patterns) that exist when defining operations in WSDL files...


One-way
- Contains only an input message.

Request-response
- Contains an input message, followed by an output message (may contain a fault message)

Solicit-response
- Contains an output message, followed by an input message (may contain a fault message)

Notification
- Contains only an output message.


I will cover messages later in this article, but I have displays the tags here (For the Add operation only) to see how operations are mapped onto messages. AddFaultMessage takes a simple string type, however, complex types (Arrays, Lists, Custom Objects etc..) may exist which require definition in the Types section of the WSDL. Lets have a look at this next...




Types

The types section defines the data types that are used by the service. If the service uses only simple types I.e. Integers, Strings etc... then the types section is not required. WSDL types can also be reused across multiple services. The types section contains one or many schema tags. XML Schema's are the typical choice, however, WSDL 1.1 is not limited to this schema type.


Example

Code Snippet

1. <types>
2. <schema targetNamespace="http://example.com/weather.xsd" xmlns="http://www.w3.org/2000/10/XMLSchema">
3. <xs:complexType name="MathInput">
4. <xs:sequence>
5. <xs:element name="x" type="xs:double"/>
6. <xs:element name="y" type="xs:double"/>
7. </xs:sequence>
8. </xs:complexType>
9. <xs:complexType name="MathOutput">
10. <xs:sequence>
11. <xs:element name="result" type="xs:double"/>
12. </xs:sequence>
13. </xs:complexType>
14. <xs:element name="Add" type="MathInput"/>
15. <xs:element name="AddResponse" type="MathOutput"/>
16. <xs:element name="Subtract" type="MathInput"/>
17. <xs:element name="SubtractResponse" type="MathOutput"/>
18. <xs:element name="Multiply" type="MathInput"/>
19. <xs:element name="MultiplyResponse" type="MathOutput"/>
20. <xs:element name="Divide" type="MathInput"/>
21. <xs:element name="DivideResponse" type="MathOutput"/>
22. </xs:schema>
23. </types>

End of Code Snippet

From our previous example, we can see that all four operations map to eight messages (one input, one output, four operations). These 8 messages map on to these eight elements defined in the Types section. Above these elements, you can see two complex types have been defined: MathInput and MathOutput. When contrasting these examples in terms on O-O programming, you can think of a complexType as an Object; and x and y as properties of this object.





Messages (Do not exist in WSDL 2.0 - Operations refer to types directly)

Messages define the input and output of operations. They either map onto an element or a type (Both defined within the Types section). As discussed within the portType section previously, the operation decides which message is used for it's input and output.

Code Snippet

1. <message name="AddMessage">
2. <part name="parameter" element="ns:Add"/>
3. </message>
4. <message name="AddResponseMessage">
5. <part name="parameter" element="ns:AddResponse"/>
6. </message>
7. <message name="SubtractMessage">
8. <part name="parameter" element="ns:Subtract"/>
9. </message>
10. <message name="SubtractResponseMessage">
11. <part name="parameter" element="ns:SubtractResponse"/>
12. </message>

End of Code Snippet

Bindings

Bindings provides specific details on how the portType section will actually be transmitted over the wire. The bindings can be made available via. multiple transports: HTTP GET, HTTP POST and/or SOAP.

You can specify multiple bindings for a single portType. I.e. Making an operation available via GET, POST and SOAP.

The following example illustrates a SOAP/HTTP binding for the MathInterface portType (Defined above)...


Example

Code Snippet

1. <definitions
2. xmlns="http://schemas.xmlsoap.org/wsdl/"
3. xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
4. xmlns:xs="http://www.w3.org/2001/XMLSchema"
5. xmlns:y="http://example.org/math/"
6. xmlns:ns="http://example.org/math/types/"
7. targetNamespace="http://example.org/math/">
8. ...
9. <binding name="MathSoapHttpBinding" type="y:MathInterface">
10. <soap:binding style="document"
11. transport="http://schemas.xmlsoap.org/soap/http"/>
12. <operation name="Add">
13. <soap:operation soapAction="http://example.org/math/#Add"/>
14. <input>
15. <soap:body use="literal"/>
16. </input>
17. <output>
18. <soap:body use="literal"/>
19. </output>
20. <fault>
21. <soap:body use="literal"/>
22. </fault>
23. </operation>
24. ...
25. </binding>
26. ...
27. </definitions>

End of Code Snippet

Binding Info
- The NAME of the binding must be given a unique name. I.e. MathSoapHttpBinding - so it can be referenced from within the WSDL.
- The TYPE of the binding must map onto the name of the portType (Interface WSDL 2.0)

soap:binding Info
- This element indicates that this is a SOAP 1.1 binding.
- The style of service is "document" (See section below on WSDL Styles)
- The transport protocol is HTTP
- The soap:operation element defines the SOAPAction HTTP header value for each operation
- The soap:body element defines how the message parts appear inside of the SOAP Body element (possible values include literal or encoded)




WSDL Styles

A WSDL SOAP binding can be either a Remote Procedure Call (RPC) style binding or a document style binding. A SOAP binding can also have an encoded use or a literal use.

Using document style in SOAP indicates that the body will contain an XML document, and that the message parts specify the XML elements that will be placed there. Using rpc style in SOAP indicates that the body will contain an XML representation of a method call and that the message parts represent the parameters to the method.

This gives you four style/use models:

RPC/encoded
RPC/literal
Document/encoded
Document/literal


The use attribute specifies the encoding that should be used to translate the abstract message parts into a concrete representation. In the case of 'encoded', the abstract definitions are translated into a concrete format by applying the SOAP encoding rules. In the case of 'literal', the abstract type definitions become the concrete definitions themselves (they're 'literal' definitions). In this case, you can simply inspect the XML Schema type definitions to determine the concrete message format. For example, the Add operation for the above document/literal binding looks like this on the wire:

Using literal definitions is much cleaner and easier for the tools to get right. Using the encoding rules has led to significant interoperability problems across toolkits. It also leads to weird situations like not being able to validate the wire-level message against the original schema definition (since it's abstract and not a true representation of the message)

More info on WSDL Styles here: http://www.ibm.com/developerworks/webservices/library/ws-whichwsdl/



Service

The service element defines a collection of endpoints/ports that expose our bindings (Listed above). Each port must be given a name and assigned a binding.

Example

Code Snippet

1. <definitions
2. xmlns="http://schemas.xmlsoap.org/wsdl/"
3. xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
4. xmlns:xs="http://www.w3.org/2001/XMLSchema"
5. xmlns:y="http://example.org/math/"
6. xmlns:ns="http://example.org/math/types/"
7. targetNamespace="http://example.org/math/">
8. ...
9. <service name="MathService">
10. <port name="MathEndpoint" binding="y:MathSoapHttpBinding">
11. <soap:address
12. location="http://localhost/math/math.asmx"/>
13. </port>
14. </service>
15. </definitions>

End of Code Snippet

Full WSDL Example (For all above examples)

Code Snippet

1. <definitions xmlns="http://schemas.xmlsoap.org/wsdl/"
2. xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
3. xmlns:http="http://schemas.xmlsoap.org/wsdl/http/"
4. xmlns:xs="http://www.w3.org/2001/XMLSchema"
5. xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
6. xmlns:mime="http://schemas.xmlsoap.org/wsdl/mime/"
7. xmlns:y="http://example.org/math/"
8. xmlns:ns="http://example.org/math/types/"
9. targetNamespace="http://example.org/math/">
10. <types>
11. <xs:schema targetNamespace="http://example.org/math/types/"
12. xmlns="http://example.org/math/types/"
13. elementFormDefault="unqualified" attributeFormDefault="unqualified">
14. <xs:complexType name="MathInput">
15. <xs:sequence>
16. <xs:element name="x" type="xs:double"/>
17. <xs:element name="y" type="xs:double"/>
18. </xs:sequence>
19. </xs:complexType>
20. <xs:complexType name="MathOutput">
21. <xs:sequence>
22. <xs:element name="result" type="xs:double"/>
23. </xs:sequence>
24. </xs:complexType>
25. <xs:element name="Add" type="MathInput"/>
26. <xs:element name="AddResponse" type="MathOutput"/>
27. <xs:element name="Subtract" type="MathInput"/>
28. <xs:element name="SubtractResponse" type="MathOutput"/>
29. <xs:element name="Multiply" type="MathInput"/>
30. <xs:element name="MultiplyResponse" type="MathOutput"/>
31. <xs:element name="Divide" type="MathInput"/>
32. <xs:element name="DivideResponse" type="MathOutput"/>
33. </xs:schema>
34. </types>
35. <message name="AddMessage">
36. <part name="parameters" element="ns:Add"/>
37. </message>
38. <message name="AddResponseMessage">
39. <part name="parameters" element="ns:AddResponse"/>
40. </message>
41. <message name="SubtractMessage">
42. <part name="parameters" element="ns:Subtract"/>
43. </message>
44. <message name="SubtractResponseMessage">
45. <part name="parameters" element="ns:SubtractResponse"/>
46. </message>
47. <message name="MultiplyMessage">
48. <part name="parameters" element="ns:Multiply"/>
49. </message>
50. <message name="MultiplyResponseMessage">
51. <part name="parameters" element="ns:MultiplyResponse"/>
52. </message>
53. <message name="DivideMessage">
54. <part name="parameters" element="ns:Divide"/>
55. </message>
56. <message name="DivideResponseMessage">
57. <part name="parameters" element="ns:DivideResponse"/>
58. </message>
59. <portType name="MathInterface">
60. <operation name="Add">
61. <input message="y:AddMessage"/>
62. <output message="y:AddResponseMessage"/>
63. </operation>
64. <operation name="Subtract">
65. <input message="y:SubtractMessage"/>
66. <output message="y:SubtractResponseMessage"/>
67. </operation>
68. <operation name="Multiply">
69. <input message="y:MultiplyMessage"/>
70. <output message="y:MultiplyResponseMessage"/>
71. </operation>
72. <operation name="Divide">
73. <input message="y:DivideMessage"/>
74. <output message="y:DivideResponseMessage"/>
75. </operation>
76. </portType>
77. <binding name="MathSoapHttpBinding" type="y:MathInterface">
78. <soap:binding style="document"
79. transport="http://schemas.xmlsoap.org/soap/http"/>
80. <operation name="Add">
81. <soap:operation soapAction="http://example.org/math/#Add"/>
82. <input>
83. <soap:body use="literal"/>
84. </input>
85. <output>
86. <soap:body use="literal"/>
87. </output>
88. </operation>
89. <operation name="Subtract">
90. <soap:operation soapAction="http://example.org/math/#Subtract"/>
91. <input>
92. <soap:body use="literal"/>
93. </input>
94. <output>
95. <soap:body use="literal"/>
96. </output>
97. </operation>
98. <operation name="Multiply">
99. <soap:operation soapAction="http://example.org/math/#Multiply"/>
100. <input>
101. <soap:body use="literal"/>
102. </input>
103. <output>
104. <soap:body use="literal"/>
105. </output>
106. </operation>
107. <operation name="Divide">
108. <soap:operation soapAction="http://example.org/math/#Divide"/>
109. <input>
110. <soap:body use="literal"/>
111. </input>
112. <output>
113. <soap:body use="literal"/>
114. </output>
115. </operation>
116. </binding>
117. <service name="MathService">
118. <port name="MathEndpoint" binding="y:MathSoapHttpBinding">
119. <soap:address location="http://localhost/math/math.asmx"/>
120. </port>
121. </service>
122. </definitions>

End of Code Snippet

References
http://msdn.microsoft.com/en-us/library/ms996486.aspx



TODO: Add more info on WSDL 2.0

Connect via. sftp to specific port

Attempts to connect via. port 115 for SFTP

Code Snippet

1. sftp -oPort=115 user@host

End of Code Snippet

SOAP - A Complete Low-Level Guide to the Technology

I have been using SOAP for a number of years now (lol), mainly SOAP based services within the .NET architecture. However, i've never really sat down and drilling into the inner workings on a lower level.


SOAP (Simple Object Access Protocol)
- Used to accomplish internet communication between systems (software, programs, web apps, services etc.)
- HTTP is supported by all internet browsers and servers.
- SOAP therefore uses HTTP to enable this communication.
- SOAP relies on XML for its messaging format.
- Simple!


Common Misconceptions
- There is no such thing as a SOAP Protocol... It's more of a schema (a subset of XML). Therefore, SOAP based web services will typically use HTTP (Port 80) or HTTPS (Port 443).


Architecture
- A SOAP request could be a HTTP GET or a HTTP POST. For example, a HTTP POST request specifies at least two HTTP headers: Content-Type and Content-Length.
- When a GET/POST request is made to a SOAP based web service, the HTTP message will detail two parts: Header information (content length/type, host etc...) and a SOAP Envelope (All in XML)...

Here is a sample HTTP POST request using SOAP...

Code Snippet

1. POST /InStock HTTP/1.1
2. Host: www.example.org
3. Content-Type: application/soap+xml; charset=utf-8
4. Content-Length: nnn
5.  
6. <?xml version="1.0"?>
7. <soap:Envelope
8. xmlns:soap="http://www.w3.org/2001/12/soap-envelope"
9. soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
10.  
11. <soap:Body xmlns:m="http://www.example.org/stock">
12.   <m:GetStockPrice>
13.     <m:StockName>IBM</m:StockName>
14.   </m:GetStockPrice>
15. </soap:Body>
16.  
17. </soap:Envelope>

End of Code Snippet

Once the service has processed the request, it will provide a response back in the following format...

Code Snippet

1. HTTP/1.1 200 OK
2. Content-Type: application/soap+xml; charset=utf-8
3. Content-Length: nnn
4.  
5. <?xml version="1.0"?>
6. <soap:Envelope
7. xmlns:soap="http://www.w3.org/2001/12/soap-envelope"
8. soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
9.  
10. <soap:Body xmlns:m="http://www.example.org/stock">
11.   <m:GetStockPriceResponse>
12.     <m:Price>34.5</m:Price>
13.   </m:GetStockPriceResponse>
14. </soap:Body>
15.  
16. </soap:Envelope>

End of Code Snippet

SOAP Envelope Architecture
A SOAP envelope details header information, message body and any fault codes or statues that may occur/exist. Full schema can be found here: http://www.w3.org/2001/12/soap-envelope



Header (Optional)
Typically, the header is used to hold security information, I.e. authentication into the target system.


Body (Required)
The message body is used to hold all information passed or received from the SOAP service. This includes: Methods, parameters, return values etc...


Example

The Request may look like this...

Code Snippet

1. <soap:Body>
2.   <m:GetPrice xmlns:m="http://www.w3schools.com/prices">
3.     <m:Item>Apples</m:Item>
4.   </m:GetPrice>
5. </soap:Body>

End of Code Snippet

Once the service has finished processing, probably by checking a database for the price of apples, the SOAP service will return the following message body...

Code Snippet

1. <soap:Body>
2.   <m:GetPriceResponse xmlns:m="http://www.w3schools.com/prices">
3.     <m:Price>1.90</m:Price>
4.   </m:GetPriceResponse>
5. </soap:Body>

End of Code Snippet

Fault Codes (Optional - Contained in the SOAP Body)
Fault codes are used to indicate error messages. They are entirely optional. If a fault code is present, it will contain the following information: faultcode, faultstring, faultactor, detail.

The following XML displays an example of a SOAP fault in a SOAP response message.

Code Snippet

1. <soap:Body>
2.     <soap:Fault>
3.         <faultcode xsi:type="xsd:string">soap:Client</faultcode>
4.         <faultstring xsi:type="xsd:string">
5.             Failed to locate method (ValidateCreditCard) in class
6.             (examplesCreditCard) at /usr/local/ActivePerl-5.6/lib/
7.             site_perl/5.6.0/SOAP/Lite.pm line 1555.
8.         </faultstring>
9.     </soap:Fault>
10. </soap:Body>

End of Code Snippet

Interoperability
So if your wondering how the heck you can utilise this technology, you already know it's used to solve communication between programs over HTTP, so here's an example...

1. You wish to create a website that displays the latest weather information on the homepage.

2. Rather than spending thousands of pounds/dollars/euro on weather equipment (and learning how to use it!), you decide it would be better to import data from an external website.

3. Rather than "Screen Scraping" (stripping weather information out of the HTML), if we could tap right into their data store, we can display the information however we would like.

4. We now need to find a weather site that exposes a SOAP service for us to consume'... Visit: http://www.service-repository.com/ this is a good public listing of popular SOAP web services.

5. You may see GlobalWeather on the homepage. They provide a nice and easy to use SOAP service to access weather information based on city and country: http://www.webservicex.com/globalweather.asmx... It exposes two methods: GetCitiesByCountry and GetWeather.

6. By clicking on one of these methods, you can view the sample SOAP request and response messages. This provides you with how your HTTP Request should be formatted for the SOAP service to understand your request. It also tells you (from the response) what to expect back from the service. Simple!


NOTE: By understanding the inner workings of SOAP, you can call SOAP services from programming languages/environments that may not DIRECTLY support web service proxies (I.e. iPhone/iOS, Python etc...). So by sending a SOAP request using a standard HTTP Request (which most languages do support), you can achieve the same result.

Solaris [SunOS] - Installing and Managing packages using the package manager

Managing packages with Solaris is easy with pkg utilities... The post discusses the basics in adding, removing and querying packages on a Solaris system.

Finding out details of installed packages

Code Snippet

1. pkginfo | grep python

End of Code Snippet

pkginfo will display a list of all packages managed by the package manager. By searching for 'python', this gives us it's package name: SMCpython... and tells us its an application.

application SMCpython python


Obtaining Packages

1. Firstly find out your processor type and your Solaris/SunOS version.

Code Snippet

1. uname -a

End of Code Snippet

SunOS servername 5.9 Generic_117171-07 sun4u sparc SUNW,Sun-Fire-V440

This tells me that im running SunOS 5.9 (Solaris v9) and using the SPARC processor type.


2. You can optionally use pkg-get or pkgutil to automatically download and install packages (This will also automatically download package dependencies). See my blog post on how to take this option!


3. Downloading packages manually (like a boss - also if its a server/system that does not have internet access). Find and download the appropriate package from one of these mirror sites using your Solaris version and processor type from step 1...

Package Download Websites
http://www.mirrorservice.org/sites/ftp.sunfreeware.com/pub/unixpackages/
http://www.mmnt.net/db/0/0/ftp.tiscali.nl/pub/mirror/sunfreeware/


Installing Packages

Once you have obtained a package to install, you will have a file similar to this: 'gcc-3.4.6-sol9-sparc-local'. We can then use the pkgadd utility to install the package into the system. You will be warned if the package has any dependencies (in which case, you will need to install these first). The following example will install version 3.4.6 of the GCC compiler for Solaris 9 [SPARC processor]...

Code Snippet

1. pkgadd -d gcc-3.4.6-sol9-sparc-local

End of Code Snippet

Removing Packages

Packages can be removed using the pkgrm utility, alongside the name of the package. To get the name of the package, you will need to use pkginfo...

Code Snippet

1. pkginfo | grep gcc

End of Code Snippet

This gives...
application SMCgcc gcc


We can then use pkgrm on the package name SMCgcc

Code Snippet

1. pkgrm SMCgcc

End of Code Snippet

Check which package owns a file

If you would like to find out which package owns a file on your system, then you can run the following command...

Code Snippet

1. pkgchk -l -p /usr/local/lib/python2.6/genericpath.pyc

End of Code Snippet

Pathname: /usr/local/lib/python2.6/genericpath.pyc
Type: regular file
Expected mode: 0644
Expected owner: root
Expected group: bin
Expected file size (bytes): 3332
Expected sum(1) of contents: 54794
Expected last modification: Sep 09 12:02:14 2009
Referenced by the following packages:
SMCpython
Current status: installed


Creating Solaris Packages
Here are a couple of good guides on creating packages for Solaris...

http://www.ibiblio.org/pub/packages/solaris/sparc/html/creating.solaris.packages.html
http://www.bolthole.com/solaris/makeapackage.html


Misc

Solaris Package Cheatsheet

MYSQL - Backup database to a file

The mysqldump utility will allow you to backup your entire database to a file (tables definitions, relationships and data). The following statement will extract the tablenamehere definition to a file called test.sql...

Code Snippet

1. mysqldump -u root -p tablenamehere > test.sql

End of Code Snippet