MYSQL - Reset root password [Unix Systems]

1. Stop the MYSQL Service

Code Snippet

1. service mysqld stop

End of Code Snippet

or

Code Snippet

1. /etc/init.d/mysqld stop

End of Code Snippet

2. Execute the following SQL (Replace 'rootpass' woith password first)

Code Snippet

1. mysqld_safe --skip-grant-tables &
2. mysql -uroot mysql
3. UPDATE user SET password=PASSWORD("rootpass") WHERE user="root";
4. FLUSH PRIVILEGES;

End of Code Snippet

3. Start the MYSQL Service

Code Snippet

1. service mysqld start

End of Code Snippet

or

Code Snippet

1. /etc/init.d/mysqld start

End of Code Snippet

Windows Users
I imagine by typing in 'services.msc' into Start > Run, you can stop the MYSQL service there, follow the steps above, then restart the service.

Disable multiprocessing in Python

I recently installed Python 2.6 onto a dev server, and I had a few problems when running a script with logging. For some reason, the multiprocessing module wasn't being recognized/installed.

Here is a quick example, if not using multiprocessing, how you can disable it within your script.

Disable Multiprocessing

Code Snippet

1. import logging
2. logging.logMultiprocessing = 0

End of Code Snippet

How To Remove Packages With RPM "Error: ... Specifies Multiple Packages "

Simple steps to remove multiple packages installed on your linux box

Code Snippet

1. rpm -qa | grep e2fsprogs*

End of Code Snippet

This gives...
e2fsprogs-libs-1.39-15.el5
e2fsprogs-libs-1.39-15.el5
e2fsprogs-devel-1.39-15.el5
e2fsprogs-1.39-15.el5
e2fsprogs-devel-1.39-15.el5


You will see a duplicate files e2fsprogs-libs-1.39-15.el5 and gets the following error
error: "e2fsprogs-libs-1.39-15.el5" specifies multiple packages


You can solve this problem with the command below..

Code Snippet

1. rpm -e --allmatches e2fsprogs-libs-1.39-15.el5

End of Code Snippet

Python - Sending emails using smtplib

Here is a snippet that will send emails to one or more recipients. It will also send the name alongside the FROM email address.

Just plug in the SMTP server details and toggle AUTH if required.


Send an email to one or more recipients

Code Snippet

1. import smtplib
2. import email.utils
3. from email.mime.text import MIMEText
4. from email.utils import formataddr
5.  
6.  
7. def emailer():
8.  
9.             # Email Settings
10.             MESSAGEBODY = 'Test Email body.'
11.             SUBJECT = 'Test Email Subject'
12.             FROM = ('Some User', 'donotreply@someurl.com')
13.             TO = ['recipient1@someurl.com', 'recipient2@someurl.com']
14.  
15.             # SMTP Settings
16.             smtpserver = 'SMTP SERVER ADDRESS'
17.             smtpport = 25
18.             AUTHREQUIRED = 0   # If you need to use SMTP AUTH set to 1
19.             smtpuser = 'foo'  # For SMTP AUTH, set SMTP username here
20.             smtppass = 'bar'   # For SMTP AUTH, set SMTP password here
21.  
22.             # Create the message
23.             msg = MIMEText(MESSAGEBODY)
24.             msg['To'] = ', '.join(TO)
25.             msg['From'] = email.utils.formataddr(FROM)
26.             msg['Subject'] = SUBJECT
27.            
28.             try:
29.                 smtpObj = smtplib.SMTP(smtpserver, smtpport)
30.            
31.                 if AUTHREQUIRED:
32.                     session.login(smtpuser, smtppass)
33.            
34.                 smtpObj.sendmail(msg['From'], TO, msg.as_string())
35.                
36.                 print "Email has been sent successfully to: " + msg['To']
37.             except Exception, err:
38.                 print "Error: unable to send error email notification. %s"  % str(err)
39.  
40.  
41.  
42. # Invoke Emailer
43. emailer()

End of Code Snippet

Python Email Examples
http://docs.python.org/library/email-examples.html

Linux - Installing and configuring Proftpd with SFTP (mod_sftp)

I recently upgraded Proftpd on a CentOS box and added mod_sftp as a static module... This post will explain what the hell I have just said, and also, how to set it setup and running. I will include gotcha's (plenty!) and some general helpful advice along the way.

We will be building from source in this guide, so I recommend grabbing that one from the download site. Download Proftpd from their website Here. Take ProFTPD 1.3.3rc1 or later as this includes mod_sftp.


Prerequisites

1. You will need to install a copy of the OpenSSL source, if you haven't already. If you have it installed, check it's in your PATH. When compiling Proftpd, it requires some header files from the OpenSSL-dev release, so this is a must!

You can download OpenSSL from Here. Grab the latest non-beta if its going to go into production. That's the rule!

Here is a guide on installing OpenSSL


2. You also need the dev libraries for zlib. These can be found Here. Install them using the supplied ReadMe file. It will be something like this...

Code Snippet

1. ./configure
2. make
3. make install

End of Code Snippet

Things to note


Modules
There are some things to note before we get onto the configuration phase of Proftpd... Proftpd has numerous modules, each which aide different aspects of the FTP server. FOr example, for FTPS we use the mod_tls module.... for SFTP, we iuse mod_sftp module... To integrate FTP user accounts with a backend mysql database, we use mod_mysql and mod_sql_mysql. etc....

Note: Some modules are not compiled by default! mod_sftp being one of them...

While there are a lot of modules included as standard, you can download other modules Here


Static Modules vs. DSO's (Dynamic Shared Objects)

Modules can either be statuc or in DSO form. A static module basically means you are compiling it into the Proftpd compilation. On the other hand, a DSO Will be a seperate physical file of the library that is loaded in dynamically. DSO's allow you to drop modules in quickly- without re-compiling Prtoftpd each time.

Read more on DSO's in Proftpd Here


Installation

1. First we need to configure the installation of Proftpd. This is where you will need to decide

- How you want your modules to be represented (Static or DSO)
- What modules you actually want to include with the installation

I will be including mod_sftp in this installation, and im going to make it static.


2. Browse to the directory where the Proftpd source is, and execute the following command... Check the file paths beforehand though!

Code Snippet

1. ./configure --prefix=/usr --with-includes=/usr/local/ssl/include --with-libraries=/usr/local/ssl/lib --enable-openssl --with-modules=mod_sftp --enable-dso

End of Code Snippet

--prefix
Where Proftpd will be installed to. I'm using CentOS as a root user, so it will be placed in /usr/sbin

--with-includes
You need to specify the include directory for the dev installation of OpenSSL. This include directory will include the development library headers which Proftpd requires to build in mod_sftp support. You can add more paths to this parameter as you wish, seperated by a colon.

--with-libraries
Similar to above but these will point to the OpenSSL libraries.

--enable-openssl
Required to enable OpenSSL for mod_sftp

--with-modules=mod_sftp
Include the sftp module. You can specify more by adding a colon after each module name.

--enable-dso
This is for DSO modules only, but I have included it as I already had some modules built with a previous version, and wanted to include them.

Note: If you wish to use DSO's (Shared Modules)... then replace --with-modules with --with-shared and ensure --enable-dso is included. Simple!


More info on compiling can be found Here


3. Produce the Makefile

Code Snippet

1. make

End of Code Snippet

4. Install

Code Snippet

1. make install

End of Code Snippet

Installation - Troubleshooting

In file included from mod_sftp.c:30:
mod_sftp.h:83:18: error: zlib.h: No such file or directory
make: *** [mod_sftp.lo] Error 1

You didn't install the zlib-dev development libraries. See prerequisites above!


/usr/bin/ld: cannot find -lcrypto
collect2: ld returned 1 exit status
make: *** [shared] Error 1

or

/usr/bin/ld: openssl/crypto.h: No such file or directory
make[1]: *** [support.o] Error 1
make[1]: Leaving directory `/src/proftpd-1.3.2b/src'
make: *** [src] Error 2


You didn't install the OpenSSL development libraries. See prerequisites above! Ensure the libraries AND headers are present.


Configuration

1. Check proftpd has registered itself correctly and that your static modules have been installed. Execute the following command...

Code Snippet

1. proftpd -l

End of Code Snippet

mod_sftp should be included in the list here.


2. Check the correct version has been installed and verify the location of the proftpd configuration file. Execute the following command...

Code Snippet

1. proftpd -V

End of Code Snippet

The version should match (Just incase an upgrade went wrong!) and you can see which configuration file Proftpd is using here... It should reside within the /etc directory of the --prefix specified when configuration the installation. I.e. /usr/etc/proftpd.conf


3. Open proftp.conf with your favourite text editor. The config file has the same syntax as Apache server configs, so it's easier to get to grips with.

View sample configurations Here

If you take a look at the Complex Virtual example, you can see how different VirtualHosts are configured.

Note: The Global tag contains settings that wiull be included across all VirtualHost's within the configuration file. Therefore, there is only ever one instance of the Global tag.

4. Take a copy of this file, and setup your Virtual hosts based on this format. You can even use the simple config file to start with. However, it is useful to understand what is possible here.

5. Save your proftpd.conf file and restart the proftpd service.
See this link for restarting services on various distros of Linux: http://theos.in/desktop-linux/tip-that-matters/how-do-i-restart-linux-network-service/

This ensures the latest config is read and is compatible before we continue.

[root@servername proftpd]# service proftpd restart
Shutting down proftpd: [ OK ]
Starting proftpd: [ OK ]


6. Keep the proftpd.conf file open and add the following for SFTP support.

Code Snippet

1. <IfModule mod_sftp.c>
2.         <VirtualHost xx.xx.xx.xx>
3.       SFTPEngine on
4.       SFTPLog /etc/proftpd/sftp/sftp.log
5.  
6.       # Configure the server to listen on port
7.       Port 2222
8.  
9.       # Configure both the RSA and DSA host keys, using the same host key
10.       # files that OpenSSH uses.
11.       SFTPHostKey /etc/ssh/ssh_host_rsa_key
12.       SFTPHostKey /etc/ssh/ssh_host_dsa_key
13.  
14.  
15.       # Configure the file used for comparing authorized public keys
16.       SFTPAuthorizedUserKeys file:/root/.ssh/authorized_keys
17.  
18.       # Enable compression
19.       SFTPCompression delayed
20.  
21.       # Allow the same number of authentication attempts as OpenSSH.
22.       #
23.       # It is recommended that you explicitly configure MaxLoginAttempts
24.       # for your SSH2/SFTP instance to be higher than the normal
25.       # MaxLoginAttempts value for FTP, as there are more ways to authenticate
26.       # using SSH2.
27.       MaxLoginAttempts 6
28.  
29.         </VirtualHost>
30. </IfModule>

End of Code Snippet

Note: Replace xx.xx.xx.xx with IP or hostname on which to accept incoming connections.

Note: Set the port for SFTP. If you have OpenSSH installed, you will either need to change the port OpenSSH uses, or change the port within this VirtualHost. I have opted for the latter, and use port 2222 for SFTP connections. You can't have two things on one port! Check what is running on certain ports using netstat.

Code Snippet

1. netstat -tulpn

End of Code Snippet

Note: The SFTPAuthorizedUserKeys attribute points to an authorized_keys file or public key authentication. Huh?? See my guide here on what it is and how to get setup

Note: You will notice the VirtualHost is wrapped within an IfModule tag. This is a condition that will granted true if the mod_sftp loaded correctly.


7. Restart the Proftpd service and test the connectivity using an FTP client.

Code Snippet

1. lftp sftp://user@server:port    (I.e. lftp sftp://sean@server:2222)
2.  
3. or
4.  
5. sftp user@server:port    (I.e. sftp sean@server:2222)

End of Code Snippet

You can also use FileZilla by setting the connectivity type to SFTP and specifying your configured port (I.e. 2222).


OPTIONAL: Module Configuration (DSO Configuration Only)

If you would like to include DSO's into the proftpd configuration, then open the Proftpd.conf file and add the following line...

Code Snippet

1. Include                         /path/to/modules.conf

End of Code Snippet

This path should point to an empty file called modules.conf. You can create this using your favourite text editor.

Now open up your newly created modules.conf file and add the following...

Code Snippet

1. # This is the directory where DSO modules reside
2. ModulePath /usr/libexec/proftpd
3.  
4. # Allow only user root to load and unload modules, but allow everyone
5. # to see which modules have been loaded
6. ModuleControlsACLs insmod,rmmod allow user root
7. ModuleControlsACLs lsmod allow user *
8.  
9. # Load the modules
10. LoadModule MODULENAMEHERE1.c
11. LoadModule MODULENAMEHERE2.c
12. LoadModule MODULENAMEHERE3.c

End of Code Snippet

The directory you specified within the ModulePath attribute should point to the directory where your DSO's reside. Now replace MODULENAMEHERE with the name of your modules.
Once you have done this, you may need to restart the Proftpd service for the changes to take effect.



Useful features and Debugging help

Proftpd Debug mode

You can place Proftpd in debugging mode using the following command...

Code Snippet

1. proftpd -nd10

End of Code Snippet

Please note, that you will have to stop the Proftpd service first, or you will get an 'address in use' error. Makes sense!

How to stop a service: http://theos.in/desktop-linux/tip-that-matters/how-do-i-restart-linux-network-service/

This is level 10 debugging and will show you connecting clients and all activity within the console output. Very useful to troubleshoot connectivity issues and auth.

Full documentation on debugging Here


Trace Logging
The mod_sftp module supports different forms of logging. The main module logging is done via the SFTPLog directive. For debugging purposes, the module also uses trace logging, via the module-specific "scp", "sftp", and "ssh2" log channels. Thus for trace logging, to aid in debugging, you would use the following in your proftpd.conf:

Code Snippet

1. TraceLog /path/to/sftp-trace.log
2. Trace scp:20 sftp:20 ssh2:20

End of Code Snippet

This trace logging can generate large files; it is intended for debugging use only, and should be removed from any production configuration.

mod_sftp Documentation Here


Logging

See the official guide on how logging can help you debug Proftpd. Logging documentation can be viewed Here

Where is proftpd?

Code Snippet

1. which -a proftpd

End of Code Snippet

Any problems or issues, please leave a comment and I will assist you where I can. I wrote this up from my notes, and im confident I have included most of the vital parts!

Shoutouts for the 200th BLOG POST!!!

Unix/Linux - Setting up public key authentication [SFTP/SSH/SCP etc]

Today I had the task of setting up public key authentication onto an existing system. The primary task was to ensure that you could send a file via SFTP without a password (password authentication). Joy!


Public key authentication - What the...?
This is basically an authentication method, just like specifying a username and a password (password auth).... The only difference is that a 'key' (some long encrypted string) is generated on a client machine, and the server is made aware of this in a config file. So when the client attempts to connect, the server will check the config, realise its the client, match it against the username, and we're in! Sounds easy I guess.... The basic aim is to get in without a password, while still maintaining security....

Let's start with the client (The machine you want to connect from)

Client Configuration

Note: Firstly, use the user which you wish to allow public key authentication for. We will be storing the keys in the $HOME directory, so make sure you've switched to the correct user from the start.

1. Generate a public and private key. Run this command...

Code Snippet

1. ssh-keygen -t dsa

End of Code Snippet

Example
[root@someserver ~]# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/root/.ssh/id_dsa): /root/.ssh/id_dsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.
The key fingerprint is:
blah:blah:blah:blah:blah:blah:blah:blah:blah:blah root@someserver

Note:
- I am using the root user here... root will be replaced by the name of your user.
- I used the same path with was hinted by the ssh-keygen utility. The keys should be generated within your $HOME dir under the .ssh directory. I would maybe rename id_dsa to something else, as this is the default name. Just a security consideration as somebody could search for your keys using the default name.
- Leave passphrase blank for getting this up and running. It just means you would have to provide the phrase in future.


In this example....
Private Key: /root/.ssh/id_dsa
Public Key: /root/.ssh/id_dsa.pub


You public key is the one you advertise... the private key, never let out of your sight! It's as good as a password to gain entry! But you know that already right?

2. Set permissions if you haven't already... security is number 1!

Code Snippet

1. chmod 700 ~/.ssh
2. chmod 600 ~/.ssh/id_rsa
3. chmod 644 ~/.ssh/id_rsa.pub

End of Code Snippet

3. Open the public key file and copy the contents. I recommand not using 'more' to copy because it puts new lines in where the console window wraps. Use something like vi text editor.

4. Keep this public key content handy because we'll need to tell the server about it!


Server Configuration

Note: I will assume you are logged in as a root user here.

1. First of all, lets enable publickey authentication! Open the sshd_config file for editing. I personally use vi. sshd_config is either in /etc/ , /etc/ssh/ or /etc/conf.d/

Mine is here: /etc/ssh/sshd_config


2. You need to enable RSAAuthentication and PubkeyAuthentication... You may need to uncomment those lines. Also, set the AuthorizedKeysFile to point to the authorized_keys file on the server. This file sits in the .ssh directory with the $HOME dir (Just like in the client config). However, we are using the root user, so it should be here: /root/.ssh or simply: ~/.ssh. Check the path is correct before you save.

Code Snippet

1. RSAAuthentication yes
2. PubkeyAuthentication yes
3. AuthorizedKeysFile     ~/.ssh/authorized_keys

End of Code Snippet

3. Save changes and close sshd_config. I usually restart the sshd service just incase.

Code Snippet

1. service sshd restart

End of Code Snippet

See this link for restarting services on various distros of Linux: http://theos.in/desktop-linux/tip-that-matters/how-do-i-restart-linux-network-service/


4. Now lets tell the server about our client's public key! Exciting right? Navigate to the .ssh directory (as discussed in step 2). Either open or create the file authorized_keys. This will contain various public keys for connecting clients. Edit this file and add the public key from step 3 in the client configuration. One entry per line if you already have content within there. Save the file.

5. Lets test!


Testing

1. Go back to your client and attempt to SSH onto your server.

Code Snippet

1. ssh username@servername

End of Code Snippet

You can also force ssh to use publickey auth (you may have various types of auth enabled).

Code Snippet

1. ssh -vvv -o PreferredAuthentications=publickey username@servername

End of Code Snippet

2. Simple SFTP test.

Code Snippet

1. sftp username@servername

End of Code Snippet

Any problems, please comment and I can assist where necessary! Have fun I guess....

Unix/Linux - Packet Analyzers - snoop and tcpdump

snoop
snoop is a very flexible command line packet analyzer included as part of Sun Microsystems' Solaris operating system.


tcpdump
tcpdump is a common packet analyzer that runs under the command line. It allows the user to intercept and display TCP/IP and other packets being transmitted or received over a network to which the computer is attached.


Examples

Troubleshoot a host

Code Snippet

1. snoop -v -d qfe0 -x0 host 192.168.1.87
2. tcpdump -i en0 host 192.168.1.87

End of Code Snippet

Exclude the host you're connected from

Code Snippet

1. snoop -x0 -d hme0 not host 192.168.1.20
2. tcpdump -i eth0 not host 192.168.1.20

End of Code Snippet

View only SSL packets

Code Snippet

1. snoop -x0 port 443
2. tcpdump port 443

End of Code Snippet

For more examples, view the man pages.

Linux - Manually and automatically adding system users

Please view this link for an indepth guide on how to add system users to /etc/passwd
http://floppix.ccai.com/adduser.html

Please Note

[root@server seantest]# passwd seantest
Changing password for user seantest.
passwd: Authentication token manipulation error

If u are running shadowed passwords there might be no entry for this user.


1. Make a backup of /etc/shadow

Code Snippet

1. cp /etc/shadow /etc/shadow.backup

End of Code Snippet

2. Delete /etc/shadow

Code Snippet

1. rm /etc/shadow

End of Code Snippet

3. Convert /etc/passwd using password convert

Code Snippet

1. pwconv

End of Code Snippet

And the same for groups...

1. Make a backup of /etc/gshadow

Code Snippet

1. cp /etc/gshadow /etc/gshadow.backup

End of Code Snippet

2. Delete /etc/gshadow

Code Snippet

1. rm /etc/gshadow

End of Code Snippet

3. Convert /etc/group using group convert

Code Snippet

1. grpconv

End of Code Snippet

Full path of service and service temporary directory when running

Execute the following command to view all processes and do a search of a service name (The service your searching for)

Code Snippet

1. ps auxwwwe | grep SERVICENAMEHERE

End of Code Snippet

This will give you back some info about the service. We are really only interested in the ProcessID (PID). This will be an integer.

Example return
root 24466 0.0 0.0 1476 280 ? S 2009 0:00 supervise sshd

PID will be: 24466

So now navigate to /proc and view the contents. You will see a directory for your PID. View the contents of this directory to see the info for the service.

The file path is located @ exe in this example: exe -> /usr/sbin/servicename

proftpd - Adding FTPS Support (mod_tls module)

This guide assumes you have proftpd installed with OpenSSL libs. If not, follow this guide

Firstly, I will quickly say....

FTPS or SFTP

People intend to mix FTPS and SFTP together, but both are actually completely differend.

FTPS is a normal FTP server but using SSL encrytion.
SFTP is a ftp kind of session over SSH (so everything is encrypted just like in SSH).


Notes
Users Guide: http://proftpd.org/localsite/Userguide/linked/userguide.html

Steps

*** Ensure mod_tls module is available within your proftpd installation.
*** Ensure you are a root user


1. Open proftpd.conf and add an include to a config file we are going to create (tls.conf). Add the following line below...

Code Snippet

1. Include         /etc/proftpd/tls.conf

End of Code Snippet

2. Now use vi to create the config file in the specified location...

Code Snippet

1. vi /etc/proftpd/tls.conf

End of Code Snippet

3. Enter the following information into the file (How to use vi)

Code Snippet

1. <IfModule mod_tls.c>
2.  
3.         TLSEngine                               on
4.         TLSLog                                  /var/log/proftpd/tls.log
5.         TLSProtocol                             SSLv23
6.         TLSRSACertificateFile                   /etc/proftpd/ssl/proftpd.cert.pem
7.         TLSRSACertificateKeyFile                /etc/proftpd/ssl/proftpd.key.pem
8.        
9.         #
10.         # Avoid CA cert and allow client renegotiation (to overcome 1.3.2c bug 3324)
11.         #TLSOptions                             NoCertRequest AllowClientRenegotiation
12.         #
13.         # Authenticate clients that want to use FTP over TLS?
14.         #
15.        
16.         TLSVerifyClient                         off
17.        
18.         #
19.         # Are clients required to use FTP over TLS when talking to this server?
20.         #
21.        
22.         TLSRequired                             on
23.        
24.         #
25.         # Allow SSL/TLS renegotiations when the client requests them, but
26.         # do not force the renegotations.  Some clients do not support
27.         # SSL/TLS renegotiations; when mod_tls forces a renegotiation, these
28.         # clients will close the data connection, or there will be a timeout
29.         # on an idle data connection.
30.         #
31.        
32.         TLSRenegotiate                          required off
33.        
34. </IfModule>

End of Code Snippet

4. Generate certificate using OpenSSL

Code Snippet

1. openssl req -new -x509 -days 365 -nodes -out /etc/proftpd/ssl/proftpd.cert.pem -keyout /etc/proftpd/ssl/proftpd.key.pem

End of Code Snippet

5. Save and close the file.

6. Now restart proftpd for the changes to take effect.

Code Snippet

1. /etc/init.d/proftpd restart

End of Code Snippet

7. Test FTPS connectivity with the server. See below...
Note: if there are any issues with the connection process, check the log file within the tls.conf file we created: /var/log/proftpd/tls.log


Testing FTPS with lftp

1. Execute following command

Code Snippet

1. lftp -u USERNAMEHERE -e 'set ftp:ssl-force true,ftp:ssl-protect-data true' SERVERNAMEHERE

End of Code Snippet

2. Enter password for user.

3. Perform a simple command...

Code Snippet

1. ls -l

End of Code Snippet