Django Setup

Been learning a new web framework since I had time on my hand last month. I thought I'd get started on web development since this seems to be where the action is headed in the near future (mobile apps look good too but couldn't afford those toys so I'm sticking with something more 'freebie').

Two of the most popular web framework in the scene are Ruby on Rails and Django. RoR is out, since I'm not keen to learn a new language just so I could try the latest/greatest web frameworks. Fortunately, Django was built on top of Python, so it was really more of applying my rudimentary python skill into another domain.

Looking for Django tutorial/howtos, I found the online version of "The Definitive Guide to Django" here which is probably the best intro for Django. 'been playing around with it and I thought I'd create own quickie reference material that I can quickly jump into without reading the voluminous manual.

Part 1: Introduction

Django is a web framework, basically what it does is do most of the grunt work of web development and provide you with plenty of abstraction (or shortcuts) to most common and frequently done task when developing web apps. For instance, you can use Django's authentication module for authenticating your web users rather than cobbling something amateurish on your own.

Django follows the MVC (Model-View-Controller) philosophy which implement a strict separation of different parts of the application (aka loose coupling) which makes it easy to change parts of the code without impacting other parts of the application. However, in Django they call it MTV (Model-Template-View). Model is your database setup while Template is generally your presentation layer (ie. html/ajax) of your web app and View is your "business logic layer" and binds your model and template.

Part 2: Installation and Setup of Django

Step 1: I use Fedora, open your terminal, go to root and run the following:

yum -y install Django mysql MySQL-python

rpm -q Django mysql MySQL-python

Step 2: Start a Django projet

mkdir ~/djcode #This is where your Django code will live

cd ~/djcode

django-admin.py startproject mysite #Create the appropriate config files

cd mysite

python manage.py runserver& #Run the development web server

firefox http://127.0.0.1:8000 #Test your Django setup

You should now have working Django setup and ready for your web development

Part 3: Simple Django Web Application

Let's create a simple web page to illustrate how Template and View (in MTV acronym) works.

Step 1:

Create a template directory where you'll place all your html files.

mkdir Templates

Step 2:

You need to tell Django where to find your templates. In your ~/djcode/mysite/settings.py find the TEMPLATE_DIRS settings and add the following:

TEMPLATE_DIRS = (

'/home/gene/djcode/mysite/Templates',

)

Step 3:

Go into the Template directory (ie. "cd Templates") and create a file named basic.html:

The "{{ nickname }}" and the "{{ date }}" inside the html file indicate that these are variables that will be defined when you 'render' this template.

Step 4:

We now need to create a view. Remember that view describe which data you see and not how you see your data. View process that data before it is presented. So in the ~/djcode/mysite directory, create the view.py file:

from django.shortcuts import render_to_response

import datetime

def current_datetime(request):

name = 'Gene Ordanza'

now = datetime.datetime.now()

return render_to_response('basic.html', {

'date': now,

'nickname': name })

If you're fairly new to web framework and not coming in from programming background, render_to_response is the tricky part. Both 'nickname' and 'date' in the basic.html are variable and so is the 'name' and 'now' in the view.py. What view.py does is call the template (in this case basic.html) through render_to_response method and then defined the variable inside. Remember: separation of different part of your web app is what you're after. So your web designer or web developer are free to mess around with their respective area of expertise without messing other parts of your application.

Step 5:

You now have to map your template to your view. Now this is where your URLconf comes in. URLconf file is something like table of contents for your web app. Basically, you go to URL address then your Django framework will map that URL to a view you've define. In turn, your view will call your template. That in essence is how your Django works in the background.

In the ~/djcode/mysite/urls.py file, add the following:

from mysite.view import current_datetime

urlpatterns = patterns(' ',

(r'^time/$', current_datetime)

)

Also, in ~/djcode/mysite/settings.py file, look for the TIME_ZONE variable and change it to whatever appropriate time zone for you. In my case, that's TIME_ZONE = 'Asia/Manila'. Voila! That's it, point your browser to the following address http://127.0.0.1:8000/time.

Part 4: Configure Database

One of the core feature of Django is ORM (Object-Relational Mapper) and this is where most web frameworks really shines. Basically, ORM enables you to link your data models in Django to the actual database (in this case MySQL). Loosely defined, our data models refer to your database tables. With Django's model, you don't have to get down and dirty with database access. Most of the data retrieval and manipulation can be done within Django as it provide a nice abstraction to your database application. For our web app, we'll create a book/author/publisher database.

Step 1:

Setup your MySQL server

service mysqld start

mysqladmin password #setup your password

mysqladmin -u root -p create testdb #create the testdb database in MySQL

Step 2:

In your ~/djcode/mysite/settings.py file, add the following:

DATABASE_ENGINE = 'mysql'

DATABASE_NAME = 'testdb'

DATABASE_USER = 'root

DATABASE_PASSWORD = 'hello' #Or whatever password you've set

Step 3:

You now need to create an app where you model and views will reside for that application.

cd ~/djcode/mysite

python manage.py startapp books

Step 4:

The command above should create a subdirectory called ~/djcode/mysite/books. Inside that directory, edit the file models.py and add the following:

from django.db import models

class Publisher(models.Model):

name = models.CharField(max_length=30)

address = models.CharField(max_length=30)

city = models.CharField(max_length=50)

state_province = models.CharField(max_length=30)

country = models.CharField(max_length=30)

website = models.URLField()

def __unicode__(self):

return self.name

class Author(models.Model):

first_name = models.CharField(max_length=30)

last_name = models.CharField(max_length=30)

email = models.EmailField()

def __unicode__(self):

return u'%s %s' % (self.first_name, self.last_name)

class Book(models.Model):

title = models.CharField(max_length=100)

authors = models.ManyToManyField(Author)

publisher = models.ForeignKey(Publisher)

publication_date = models.DateField()

def __unicode__(self):

return self.title

Step 5:

At this point, the testdb database is still empty. We've defined 3 models (Publisher, Author, Books) that correspond to your database tables (that we haven't created yet). We need to activate these models to create the actual tables. In the settings.py file, find the INSTALLED_APPS setttings and comment out the five default apps enabled there (ie. 'django.contrib.admin') by prepending a '#' before the app name. And then add the 'mysite.books' app in the INSTALLED_APPS, so it should look like this:

INSTALLED_APPS = (

# 'django.contrib.auth',

# 'django.contrib.contenttype',

# 'django.contrib.sessions',

# 'django.contrib.sites',

# 'django.contrib.auth',

'mysite.books',

)

In the MIDDLEWARE_CLASSES comment out the middleware apps as well.

Step 6:

The 'mysite.books' is the web app that we're working on. We now need to validate/check our settings, just to be sure that we don't have any typo or illegal settings. In the ~/djcode/mysite directory, run the following command:

python manage.py validate

If no error came up, then you can now commit the Django model to the MySQL database,

python manage.py syncdb

When you run the command above, it should prompt you to also create a root account and password.

That's it! You have now created and sync the following tables: books_publisher, books_author, books_book.

Part 5: Set-up of Admin Site

Like the ORB feature, the Admin Site is one of the core feature of Django and what makes developing web apps for Django easier and more productive. With Admin Site, writing boilerplate code for your web app infrastructure is taken care for you so you can immediately add, change and view content for your web site and avoid getting bogged down with the usual repetitive code of making production-ready interface for your web site.

Step 1:

In the settings.py file, uncomment back the settings for INSTALLED_APPS and the MIDDLEWARE_CLASSES to enable the Admin Site.

Step 2:

Sync your database. The settings that were uncommented above are packages that need to sync to the database. In your ~/djcode/mysite directory, run the following:

python manage.py syncdb

This will prompt you to create a root account for your Admin site.

Step 3:

In urls.py, uncomment the following code for the Django admin:

from django.contrib import admin

admin.authodiscover()

urlpatterns = patterns(' ',

# . . .

(r'^admin/', include(admin.site.urls)),

# . . .

)

Step 4:

Point your browser to the http://127.0.0.1:8000/admin and you should get the following web page:

Step 5:

Try to log in using the root account you've created when you run the "python manage.py syncdb" and you should get the following web pages:

Try to create another account, change their attributes, play around and get comfortable with the Admin Site feature.

Step 6:

We'll now hook our models to the Admin Site by creating the admin.py file in the ~/djcode/mysites/books directory (ie. the books/ directory was created in Part 4). In our admin.py, add the following:

from django.contrib import admin

from mysite.books.models import Publisher, Author, Book

admin.site.register(Publisher)

admin.site.register(Author)

admin.site.register(Book)

Restart your runserver to let the development web server read the file. Go back to your Admin site and log in and you should now see the following:

At this point, you can now add or change data for your Publisher, Author, and Book database. Now, how difficult was that?

Part 6: Creating Forms

To extend our simple Publisher/Author/Book database, we'll create a rudimentary search capability that let users search books by title.

Step 1:

We'll need to create an HTML front-end for the search form. In the Template directory, create a simple html search form.

Step 2:

Create the search view that will call our html search form. In your ~/djcode/mysite/books/view.py file, add the following:

from django.shortcuts import render_to_response

def search_form(request):

return render_to_response('search_form.html')

Step 3:

In your urls.py file, map the html searh_form to the Django view.

from mysite.books import view

urlpatterns = patterns(' ',

# . . .

(r'^search-form/$', views.search_form),

# . . .

)

Step 4:

If you try to check the url http://127.0.0.1:8000/search-form, you'll get an error since we haven't yet coded the /search/ view yet. So in your view.py, add the following:

from django.http import HttpResponse

from django.shortcuts import render_to_response

from mysite.books.models import Book

def search(request):

if 'q' in request.GET and request.GET['q']:

q = request.GET['q']

books = Book.objects.filter(title__icontains=q)

return render_to_response('search_results.html', {'books':books, 'query':q})

else:

return HttpResponse('Please submit a search term.')

Step 5:

We now need to pass the books object to another template called 'search_results.html' that will display our search results. In your Template directory, create the search_results.html:

Step 6:

Now visit http://127.0.0.1:8000/search-form/ and search for both existing and non-existing titles in your database. And that's it!

If you've read this far, then you really ought to read the more concise, better written, in it's original form found here :-) If you have questions though, feel free to submit a comment and I'll try to help out.

CVS error messages: Failed to create lock directory

I've been using CVS for a few days now to keep track of my revisions in my Django scripts. Prior to CVS, I've mostly been making copies/annotating/numbering my scripts before doing any modifications (very tedious, very unwieldy). So setting up CVS repo took out the grunt work of tracking my code.

CVS compared to it's more illustrious counterparts, is old school if not the most ancient revision control system of the lot. But for on-and-off weekend coding projects that need simple versioning system, CVS it is.

Anyways, all nice and good, except that I've been getting this irritating error messages every time I cvs into my repository:

Error Messages when trying to run "cvs update" from another user.
cvs update: Updating CVSROOT
cvs update: failed to create lock directory for `/cvs/CVSROOT' (/cvs/CVSROOT/#cvs.lock): Permission denied
cvs update: failed to obtain dir lock in repository `/cvs/CVSROOT'
cvs [update aborted]: read lock failed - giving up

Turns out that when I first "cvs import" my files, the default permission was set to the user who initially created the repository. So the problem is really of 'gene' user not having write access to the files 'owned' by the first user who created the repo. The solution I found here and here indicated that you need to setup proper group ownership for the repo. That means make 'gene' user and all other users who'll be using cvs be part of the group that will owned the repo. In this case, just create a group called 'developers' or any other group name, and have all subdirectories beneath the root CVS directory owned by this group then setgid all the subdirectories (ie. chmod -R g+ws). Voilà! works beautifully.

There's heaps of open-source software revision control system, if you want a Fedora/CVS setup, here's a good tutorial.

How to convert different video file format to MP4 format for your PSP/Ipod

So we're going out-of-town this weekend, it's a two-hour drive and my two nephews, 5 and 6 years old, are coming with us.

Question: What are the odds that these two boys will willingly share and wait for their turn for their PSP toy.

Answer: Yeah, right. Battle royal, wailing and crying is more like it for that 2 hours long drive.

So what's an intrepid uncle ought to do? Why find a reasonable middle-ground that both these boys can agree on, of course.

And how do you propose to do that? Hehe, 'got a plan, boy. Get their favorite movies/TV shows from bittorrent (ie. boy with a tacky watch and that small Japanese robot boy that can fly) and put them on their PSP. Tada! Good, no?

Now this is more like Part II of my last blog (ie. how to rip videos from your DVDs), since you can also use dvdrip to encode the file to different format except that I couldn't find a container format for H.264/MPEG-4 (which is the format of choice for handheld devices). I tried mencoder like real men do, but it was effin messy and couldn't get the command-line options right :-( There has to be easier to do it.

As always, I assume you're using your friendly Fedora Linux distro. Also, bear in mind that every time you convert from one format to another, the quality of your video gradually degrades (not that you'll notice it when watching it on your PSP).

And so without further ado, here's how you can convert those avi/wmv/mpeg files from TPB into something you can watch on your PSP/iPhone or any handheld devices.

Avidemux rock!!

===

Step 1: Log in as root and from the console, run the following command:

# yum -y install avidemux

Step 2: Log in as regular user and from the console, run the following command:

$ rpm -q avidemux
$ avidemux

You should see the following Avidemux window:


Step 3: Open the file video file you wish to encode to another format. In the menu, click File->Open and then look for the video file you would like to convert.



Note: If you get a message that says something like "Index is not up date: You should use Tool->Rebuild frame. Do it now?". Just click "Yes".

Step 4: In the menu, click Auto->PlayStation Portable(H.264). You should see the following window:


Don't forget to

Step 5: Save the file, click File->Save (or just click the Save button). The Save window should pop up.


Type the name of your new video file. Don't forget to include the .mp4 extension to the name of the file.

Step 6: Once you click on the Save button, encoding should now start, you can now sit back and wait. Depending on fast your workstation is, and how large the file you want encode, this might take a while.


Once it finished, you should get the following pop-up message box.


Step 7: Now you can hook-up your PSP to your computer and copy the .mp4 file to the Video folder of your PSP.

If you're video editing needs are modest, you can also use Avidemux for cutting/deleting some scene, or join two or more video clips into a single video file, or add borders and subtitles to your videos.

Enjoy!!

How to rip video CD/DVDs using dvdrip tool

First a few rants...

Don't you hate it when you commissioned someone to video tape an event and all they give you is a DVD copy of the event. What if you wanted your friends and relatives to watch this video, do you give each one of them a DVD copy? Or what if you want to store it on your handheld devices like mobile phone or PSP? Or as is most likely the case, you just want to post it on Youtube and share it? What do you do? No can do?

I mean, it's your wedding day or your first childs' baptismal or any of those special events, you definitely want someone professional to do it right? You'd even be glad to pay for it. But then, you'd also want to share the finish product with as many friends/relatives/colleagues as you can.

I suppose these folks also want to protect their intellectual property rights, and I don't begrudge them that (never mind those DVD movies you legally paid for). But since it's a personal video of you/your friends/relatives, and that you're paying for it, then it's only fair that you should be able to ask for a mpeg/avi file version of it that you can easily distribute to others.

Hehe, I'm so emo, I don't even know why I'm so work up over DVD movies when ripping them is dead easy, if you have the right tool that is. Eniways, here's how you would rip your DVD movies.


Step 1: Log in as root and run the following command:

# yum -y install dvdrip ffmpeg


Step 2: As regular user, type the following from the command line:

$ rpm -q dvdrip ffmpeg
$ dvdrip &

Note: The first time you run the dvdrip utility, it will prompt you to set your preferences.


Step 3: Notice the red highlighted text that says ".../dvdrip-data not found: NOT OK"? Well, you need create this directory. The quickest would be jump back in to your command-line prompt and type the following:

$ mkdir ~/dvdrip-data

Restart your dvdrip app and you should now get the following window image:


There's a number of settings you can fine-tuned but for now stick with the default. It works fine.

Step 4: Now you need to create a new project.

• In menu, click File->New Project. Type your project name, in my case it's skydiving. The rest of text field (ie. VOB directory, AVI directory, etc) will be automatically filled up.

You should now get this window:


Don't forget to click on the "+ Create project" button.

Step 5: In the section "Choose a ripping mode", make sure that you check "Copy data from DVD to hard disk before encoding".


Step 6: You should now go to the tab "RIP Title".

• In this tab, there's a button called "Read DVD Table of Contents", click this button to scan the files in your DVD.
• Once it finished scanning, below that, there should be a button called "RIP Selected Titles(s)/Chapter(s)". Click that button.
  

Step 7: Rip 'em boy!

• Click on the "Transcode" Tab
• Now, there's probably hundred of buttons and settings here that you can tuned and play around with. But only one setting is of really practical use to us. That's the "Container Options" which is already set to AVI container which is almost always what you want since this probably the de-facto video format in computers.
  
• Find the 'Operate' section and then click on the "Transcode" button. And off you go!

That's it. Now wait and sit back as this might take a while depending on how large DVD movie is and how fast your computer is.

Step 7: The avi file that you rip from your DVD should be in your ~/dvdrip-data/project-name/avi/ directory. Where the 'project-name' is that name you used during the initial setup.

Success!!

Use your VLC, Mplayer, Totem Movie Player, or even Xine to watch your newly ripped avi file.

Setting up Fedora Directory

An old colleague from my Sykes call center days ask me if I can set up a Fedora Directory Server and walk through his team through the whole installation and configuration. This was last Feb or March, I thought I'd gather up the notes I wrote and put them here. This will make a nice blog entry, hehe. Besides, you never know when you'll be ask to setup another Directory server and I'd hate start from scratch again. Just a heads-up though, this is long entry!

====

Distro: Fedora 8
Server: Fedora Directory Server 1.1

I. First off, some really basic intro:

LDAP (Lightweight Directory Access Protocol) is client-server protocol for accessing directory service. A directory server provides a centralized directory service for your network that can integrate wide variety of information.

Fedora Directory Server is a secure, highly scalable, robust LDAP server implementation of Red Hat and was derived from the original slapd directory server work done by UM.

II. Installation and General Fedora Directory Server Usage

1. Clean installation of Fedora 8.
   
   Note: most packages required for installing Fedora Directory Server are hosted in Fedora Repository and would require Internet access on the server.
   
   
2. Install a Java JRE, on Fedora 8 you can use IcedTea Java
   
   yum -y install java-1.7.0-icedtea
   
   
3. Setup Fedora DS yum repo,
   
   cd /etc/yum.repos.d/
   wwget http://directory.fedoraproject.org/sources/idmcommon.repo
   wget http://directory.fedoraproject.org/sources/dirsrv.repo
   
   
4. Install Fedora Directory,
   
   yum -y install fedora-ds
   
   
5. Initial setup to create an instance of the directory server
   
   cd /usr/sbin/
   ./setup-ds-admin.pl
   
   Note: Choose "Typical Installation". Also, most installation setup options
   are reasonably set, so you can accept default options.
   
   
6. Install the remote management console for managing Fedora Administration
   Server.
   
   yum -y install fedora-idm-console
   
   
7. Install the command line tools for accessing Fedora Directory Server
   
   yum -y install mozldap-tools
   
   Note: The openldap-clients package provide similar tool functionality for
   accessing traditional OpenLDAP servers.
   
   
8. Starting the Fedora Directory Server and Administration Server
   
   service dirsrv start
   service dirsrv-admin start
   
   Note: When starting the dirsrv the first time, specify the directory
   instance. To automatically start the directory services, run the following
   command:
   
   chkconfig dirsrv on
   chkconfig dirsrv-admin on
   
   Files for Fedora Directory Server can be found at,
   Log Files: /var/log/dirsrv
   Config Files: /etc/dirsrv/
   Database: /var/lib/dirsrv/slapd-instance
   Client Tools: /lib/usr/mozldap
   
   

III. Background on Directory Entries

• The directory information tree (DIT) mirror the tree model used by most
  filesystem, with the tree's root appearing on top of the hierarchy.
  
• The entry is an object that represent a particular information in directory tree (ie. person in your organization, printer in network). It is stored in a hierarchical structrue in the directory tree. An entry is defined in LDIF file.
• LDIF file is standard text-based format. Each entry in LDIF file is represented by attributes and their values.
• Schema defines the attributes type that each entries can contain. Standard schema can be found in /etc/dirsrv/schema directory.
  
  

IV. Creating entries in Fedora Directory Server Console

• Starting the Fedora Server Console
  
  fedora-idm-console -a http://localhost:9830
  
  
• Create Organizational Unit under root directory
  1. Servers and Applications ->Directory Server-> Directory
  2. Choose the root suffix and right click.
  3. Choose New->Organizational Unit
  
  
• Adding 'Users' in Organizational Unit
  1. Servers and Application->Directory Server-> Directory
  2. Right click the appropriate Organizational Unit
  3. Choose New->User
  
  
• Importing data from the Directory Server Console
  1. Servers and Applications ->Directory Server->Open->Task
  2. In the Import Database dialog box, enter full path
  3. Go to Directory tab to verify if data was successfully imported.
  
  
• Modifying entries in the Directory
  1. Servers and Applications->Directory Server->Open->Directory
  2. Right click on the entry you wish to modify
  3. Choose Advanced Properties
  4. Choose Attribute you wish to modify
  
  
• Deleting entries in the Directory
  1. Servers and Applications->Directory Server->Open->Directory
  2. Right click on the entry you wish to delete
  3. Choose delete option
  
  

V. Using LDIF Statement to Add/Modify/Delete Entries

• Create Organizational Unit under root directory
  
  ldapmodify -v -a -D "cn=directory manager" -h <hostname> -p <port> -f
  <file.ldif> -w -
  
  
• Create user account under Organization Unit
  
  ldapmodify -v -a -D "cn=directory manager" -h <hostname> -p <port> -f
  <users.ldif>
  
  
• To delete Directory entries
  
  ldapdelete -D "cn=directory manager" -h <hostname> -p <port>
  "uid=u1research,ou=research,dc=example,dc=com" -w -
  
  Note: You can only delete entries at the end of branch. You cannot delete
  entried that have sub-entries.
  
  
• Modify Directory entries
  
  ldapmodify -v -D "cn=directory manager" -h <hostname> -p <port> -f
  <file.ldif> -w -
  
  

VI. Managing Entries with Group, Roles

• Groups are mechanism for associating entries into a list.
  
  
• Roles is another entry grouping mechanism, it enables you to determined role
  membership as soon as an entry is retrieved from the directory.
  
  
• Creating Groups:
  1. Servers and Applications ->Directory Server->Open->Directory.
  2. In Menu, Object->New->Group
  3. Add group name in General folder and members in Member folder.
  
  To list down members for certain group,
  
  ldapsearch -v -D "cn=directory manager" -h <host> -p <port> -b
  "dc=example,dc=com" "cn=<name>" -w -
  
  
• Creating Roles:
  1. Servers and Applications ->Directory Server->Open->Directory.
  2. In Menu, Object->New->Roles
  3. Add group name in General folder and members in Member folder.
  
  To list specific Roles for user,
  
  ldapsearch -v -D "cn=directory manager" -h <host> -p <port> -b
  "dc=example,dc=com" "uid=<userid>" \* nsRole -w -
  
  To find all members of a particula role,
  
  ldapsearch -v -D "cn=directory manager" -h <host> -p <port> -s sub -b
  "dc=example,dc=com" "(nsRole=cn=,dc=<name>,dc=lt;name>)" dn -w -
  
  

Note: The functionality of groups and role mechanism overlap somewhat. Groups mechanism are standard-based, it is interoperable with most client and LDAP servers.

Roles mechanism is generally more efficient to use for applications as it reduce client complexity but it is more resource-intensive on the server side.


VII. Access Control

Fedora Directory Access Control defines the mechanism on how a user can access
Directory information. Access Control Instructions (ACI) are defined as attributes of entries. The three main parts of ACI are:

• Target, specify the entry, attributes for which you want to control access.
• Permission, specify the type of access that is allowed or denied.
• Bind Rule, identify the set of users to which ACI applies.
  

Creating ACI
1. Servers and Applications ->Directory Server->Open->Directory.
2. Choose the object you wish to create an ACI
3. Right click and choose Set Access Permission
4. In the Access Control Editor, set the name for ACI entry
5. In the 'Users' tab, add members (could be individual/group/roles)


VIII. Centralized Linux Authentication

• A more robust and secure alternative to using centralized authentication system through NIS.
  
  
• User accounts information are stored in Directory server for retrieval during authentication from client side.
  
  
• Home directories of the user resides in the Directory server and exported to the client side.
  
  
• To setup Fedora Directory as authentication server for client.
  
  
  1. Create the Linux user account in the server. Take note of uid and gid and /home directory for the account.
  2. In the Directory, go to "Users and Group" tab.
  3. Click on Create->User
  4. Select the Organizational Unit to put the user in and create the user account.
  5. In the "Posix User", fill in the account info based with UID, GID, Home Directory.
  6. Export(NFS) the home directory.
  7. In client side, configure authentication to use the Directory server.
  8. In the client side, edit /etc/auto.master file and add the following:
     
     /home /etc/auto.guests --timeout=60
  9. In the client side, edit /etc/auto.guests file and add the following:
     
     * -rw,soft,intr :/home/&
  10. Set autofs to automount home directories from the server
      chkconfig autofs on
      chkconfig nfs on
      service autofs start
      service nfs start
      
      

Additional Notes:

Tools under openldap-clients are not supported for Directory Server
operations. For best results with Directory Server, use tools in
mozldap-tools. Tools in this package are found in /usr/lib/mozldap
directory.

Alternatively, you can use Kontact, a GUI tool for accessing LDAP server.
Kontact is included in the kdepim rpm package.

Directory Server Gateway/Phonebook is simple web-based application that
provides search/query/update interface for directory server data but is
currently not available for version 1.1 Fedora DS.

Python Script to Pull Out MP3 Metadata

I have these number of MP3 files all lump together in a single directory that I was going to categorize and sort out over the weekend. Easy does it really, just create the directories (ie. Acoustic, Rock, RnB, etc) and then move them over to these directories.

Problem is, I have over 700~ mp3 files, and most of them have non-descriptive one word title. Some I knew right off the bat, but most of them I don't (I use 'random' mix option when playing them). Now, for me to listen to each one of them and categorize them to properly would've taken me more than one weekend.

And so I thought, wouldn't it be cool to write a small Python script to pull out the meta data (ie. artist, album, year, etc) and just based it from there? Yep, what a better way brush up on your non-existent programming fu than to write a small Pytho utility :-) And so here's the script I wrote, not the most elegant but it works (sorta :-))

Note: For reference, check out Dive Into Python book (chapter 5-6). That's where I got the baseline code. But now (except for stripnull function), the code hardly resemble the original one.

#!/usr/bin/python
# Author: Gene Ordanza
# Email: gene.ordanza AT gmail.com
# Description: Ugly hack on how to pull the metadata out of mp3 files that uses
#        the ID3v1 TAG formatting scheme.  mp3meta will also create a file called
# MetaFile.txt in your current directory.
# Usage: mp3meta <directory>
# Where: <directory>  is optional. If no directory was given, mp3meta
# looks in the current directory and walks through all subdirectory.
# Note: A great reference on how to pull ID3v1 data from mp3 file can be
# found on Chap 5-6 Dive Into Python book Chap 5-6, you can also find it online
# at http://diveintopython.org/

import sys
import os

def stripnulls(data):
 string = data.replace('\00','').strip()
 return string[:23]

def writeMetatoFile(mp3ObjectList):
 filename = 'MetaFile.txt'
 line_width = 80
 field_data = '%-5s%-24s%-24s%-24s%-4s\n'
 f1, f2, f3, f4, f5 = ('\nNo.', 'Title', 'Artist', 'Album', 'Year\n')

 if os.path.exists(filename):
  file = open(filename, 'a')
 else:
  file = open(filename, 'w')
  file.write('%s%s' % (' '*31, 'MP3 File Metadata\n'))
  file.write('=' * line_width)
  file.write(field_data % (f1, f2, f3, f4, f5))
  file.write('-' * line_width)
  file.write('\n')

 for mp3file in mp3ObjectList:
  file.write(field_data % (mp3file['count'], mp3file['Title'],\
   mp3file['Artist'], mp3file['Album'], mp3file['Year']))
 file.close()


class MP3Meta(dict):
 count = 0
 def __init__(self, name):
  self['name'] = name
  self.__class__.count += 1
  self['count'] = MP3Meta.count

 metaData = { 'Title' :  (3,  33, stripnulls),
  'Artist':  (33, 63, stripnulls),
  'Album' :  (63, 93, stripnulls),
  'Year'  :  (93, 97, stripnulls),
  'Comment': (97, 126, stripnulls),
  'Genre' :  (127, 128, ord)}

def usage():
 sys.stderr.write("""
Usage: mp3meta [directory]
Where [directory] is optional.  If you did not specify one, it will
start at your current directory and traverse all subdirectory.
 """)
 print

def getMetaData(listofMP3):
 mp3ObjectList = []
 for filename in listofMP3:
  mp3file = MP3Meta(filename)
  try:
   metafile = open(filename, 'rb', 0)
   metafile.seek(-128, 2)
   mp3data = metafile.read(128)
   if mp3data[:3] == 'TAG':
    for tag, (start, end, cleanup) in mp3file.metaData.items():
     mp3file[tag] = cleanup(mp3data[start:end])
      mp3ObjectList.append(mp3file)
   else:
    (null, title) = os.path.split(filename)
    (truncated, null) = os.path.splitext(title)
    mp3file['Title'] = truncated[:23]
    mp3file['Artist'] = ''
    mp3file['Album'] = '** Non-ID3v1.0 Format **'
    mp3file['Year'] = ''
    mp3file['Comment'] = ''
    mp3file['Genre'] = ''
    mp3ObjectList.append(mp3file)
   metafile.close()
  except IOError: pass
 writeMetatoFile(mp3ObjectList)

def findMP3(dummy, dirname, fnames):
 extension = '.mp3'
 mp3List = []
 for fname in fnames:
  if fname.endswith(extension):
   path = os.path.join(dirname, fname)
   mp3List.append(path)
 getMetaData(mp3List)


def main():
 if len(sys.argv) == 1:
  directory = os.path.abspath('.')
  os.path.walk(directory, findMP3, None)
 if len(sys.argv) == 2:
  directory = sys.argv[1]
  if os.path.isdir(directory):
   os.path.walk(directory, findMP3, None)
  else:
   usage()

if __name__ == '__main__':
 main()

Quickie Mail Server Setup - Part II

We'll continue were we left off from Part 1, make sure that the www.exampledns.com and the "mx" record for the "exampledns.com" domain is resolving properly. We'll be setting up a secure IMAP server accessible either through any of the mail user agents (ie. Thunderbird, Evolution, etc) or a Web-based email system.

We'll be using OpenSSL package (should be installed by default) in our mail server setup. OpenSSL comes with a versatile tool for generating private/public keys and certificates. The idea is to encrypt (and sign) our data packets using OpenSSL cryptographic libraries before sending them over the network . OpenSSL is an implementation of SSL/TLS functionality, it mostly gained prominence for securing transaction for e-commerce web sites (ie. banks and web retailers like Amazon) but can also be use for host of other services.

SSL/TLS protocol works by means of PKI (Public Key Infastructure). Basically, in PKI, you have a private key and certificate/public key, PKI enables users to exchange these keys and certificates securely (the server initially send a certificate to the user to be authenticated, usually via third-party Certificate Authority CA). Once both sides have verified and exchange keys, communication is encrypted using these certificates.

NOTE: When dealing with PKI setup and to fully appreciate how SSL/TLS works, make sure that you have a passing familiarity on how asymmetric cryptography works.

There a 3 ways we can setup PKI

• Using commercial Certificate Authority (CA) such as VeriSign, Thawte (there's free trial version available from VeriSign if you want to use one).

• Web of Trust popularize by PGP/GnuPG. You can get free certificates from organization such as CAcert, a community-driven CA. If you don't mind the hassle (ie. users are verified) this is a good alternative to commercial CA.
  

• Generate our self-signed certificate. Since this is the most convenient for us, we'll simply generate our own certificate.
  

With those out of the way, we'll now setup an encrypted mail service:

Step 1: Generate a self-signed certificate. We have two option here. We can use the Dovecot script /usr/libexec/dovecot/mkcert.sh or the Fedora /etc/pki/tls/certs/Makefile config (both script uses openssl behind the scene). Both automate the whole process of creating self-signed certificate for us but the Fedora Makefile script gives us more flexibility in creating certificates.

  cd /etc/pki/tls/certs
make dovecot.pem

Using make utility, we generated the dovecot.pem file that contains the private key and the certificate.

Step 2: Dovecot Configuration. Edit the /etc/dovecot.conf file and add the path for your private key and your certificate.

protocols = imaps pop3s
ssl_cert_file = /etc/pki/tls/certs/dovecot.pem
ssl_key_file  = /etc/pki/tls/certs/dovecot.pem

Save and exit and then restart the Dovecot service. That's it!

Step 3: Test your encrypted IMAP service. Point your email client to your newly encrypted mail server, and that's it. I used claws-mail, it auto-negotiate the exchange of certificates. If you're using Thunderbird, Evolution or Kmail, you might need to manually enable the settings for SSL/TLS.

Step 4: Setting up Web-Mail. This is one of the few times that we'll install from the source rather than 'yum' install the package from Fedora repo (I still get a kicked out of installing from the source from time to time).

• Get the Squirrelmail source here.
• Install the Squirrelmail source.
  

Setup the Squirrelmail directory:

  mkdir /usr/local/src/squirrelmail
cd /usr/local/src/squirrelmail
mkdir data temp
chgrp apache

Note: Traditionally, /usr/local/src/ directory is where you install 'source' programs. The 'data' and 'temp' directory is where Squirrelmail will place your data and email attachment. And finally, 'apache' should be set as the group owner of the these directories.

Unpack the Squirrelmail source and run the Squirrelmail config tool:

  mv squirrelmail-X.Y.Z-tar.gz /usr/local/squirrelmail
cd /usr/local/squirrelmail
tar -xzvf squirrelmail-X.Y.Z-tar.gz
mv squirrelmail.X.Y.Z   www
cd  www/config
./conf.pl

Step 5: Setup Apache. In your /etc/httpd/conf/httpd.conf file, add the following settings:

  Alias /webmail  /usr/local/src/squirrelmail/www

<Directory  /usr/local/src/squirrelmail/www>
 Options Indexes
 AllowOverride  none
 Order  allow,deny
 allow  from  all
</Directory>

Restart the Apache httpd service. Fire up your browser, point it to http://www.exampledns.com/webmail website and log in.

Note 1: Squirrelmail is written in Php . If you're encountering Php-related errors, check if Php package is installed.

Note 2: To avoid overly-complex, error-prone, multiple certificate-enabled server, revert Dovecot daemon back to using simple imap/pop3 protocol before setting-up an "https://" web server.

Step 6: Generate a server key and certificate for your web server (mod_ssl module handle the encryption for Apache and is normally bundled in).

But first, note that Fedora already comes with it own private key and certificate for Apache out of the box. And also Makefile generate them in pre-define directory (ie. /etc/pki/tls/{private/certs}/ ). If you would like to generate a new key/cert, you either have to rename/delete/move these old files.

rm -f /etc/pki/tls/certs/localhost.crt
rm -f /etc/pki/tls/private/localhost.key
cd /etc/pki/tls/certs
make genkey
cd /etc/pki/tls/private

By default, when you created the localhost.key it will prompt you for a pass phrase which is very annoying when you need to restart your Apache. To remove this pass phase after generating a new localhost.key,

cd /etc/pki/tls/private
cp localhost.key localhost.key-copy
openssl rsa -in localhost.key-copy -out localhost.key

We then create the certificate:

cd /etc/pki/tls/certs
make testcerts

A new localhost.crt will be generated. Restart you Apache and that's it! Point your browser to your new certificate-enabled web server and your browser should prompt you whether to accept

In Part III, we'll setup SpamAssassin and Anti-Virus using Clamav and Mimedefang.