RegistrySizeLimit…who knew?

This past Friday we pushed out 30 new lab machines. These machines were imaged the same way we image the rest of our labs. We noticed that several applications failed to install, and Carson began troubleshooting that. Turns out, due to the number of installed applications we ran out of space in the Windows Registry. Call us noobs if you like, but really…there is a max?

We found a few articles on the subject and looks like this was a setting that started for Windows NT. Here are a few of the articles we found:

We set ours to 0xffffffff which places it at approximately 4gb. The way I understand it is if this value is over 80% of the amount of paged pool, then Windows set’s it at the 80%. Unset, I believe Windows will automatically adjust this for you, but I’m not really sure on that. I rolled this fix out as a GPO Preference to only lab computers.

The issue presented itself in a few ways, first Windows Updates was broken, additionally multiple .msi installations failed with cryptic error codes.

Drupal to Django Migration

To be honest I completely ripped off Carson, he has been talking on and off about this for a while. I decided about a week ago that I would give it a stab, and so started searching Google for answers. Surprisingly there weren’t too many answers that could be found that worked or made sense. So I decided the first thing I needed to know how to do was setup a site with Django. Two friends of mine, Nick and Carson are really amazing developers. Nick wrote a Django blog app that we use at the School of Engineering and Carson wrote a Django form app that we use as well. So I figured I’d build a site using Nick’s blog and Carson’s forms and if I got stuck pester them until they helped me figure out what I messed up.

 

I am most definitely a Windows guy, and I’m really not ashamed of it, but I also really enjoy doing things just outside my comfort zone. Drupal was about as easy a setup as you could get and has worked well and I’ve had no complaints, but I’m bored with it now. So I started reading and messing around with stuff on my laptop. I started out setting up a VM with Ubuntu 10.04 Server, then I proceeded to install the following packages:

sudo apt-get install apache2 libapache2-mod-wsgi python-imaging subversion mysql-server-5.1 python-mysqldb

 

That got me nearly everything I needed, Ubuntu 10.04 already can with Python 2.6, so I just needed to grab the web server, database server and supporting libraries. I believe on the VPS host I’m using I need one of the Apache2-mpm packages but I haven’t checked that yet. The next thing I did was grab the code I needed. So because Nick and Carson keep giving me a hard time about it I’ll be grabbing the release version of Django 1.1.1, the most recent release of Nick’s blog and the most recent release of Carson’s formunculous.

wget http://www.djangoproject.com/download/1.1.1/tarball/

wget http://sourceforge.net/projects/formunculous/files/formunculous-2.1.0-final.tar.gz/download

wget http://code.photodwarf.org/downloads/python/django-blog/django-blog-1.2.tar.gz

 

After you extract them into their respective directories you will want to install each. You need to run the following command from within each project’s directory. You may get a prompt during the Formunculous install that you will need some additional tools, don’t worry it will grab them for you.

sudo python setup.py install

 

There are several different ways to lay out your sites, and the one I seem to have settled on is as follows:

/sites

/site-1

/scripts

/htdocs

/media

/templates

/site-2

 

The way I’m going to progress from here is in three parts. The first part involves setting up a dev site, a database for it and then importing your database from drupal into the blank dev database. I’m specifically working with MySQL but I would imagine it would be similar steps if you were working with Postgres. The second part will be copying the data from the drupal tables into the Django tables, again I’m specifically looking at my blog posts as well as my books. Finally dumping the Django tables out to a file, creating a fresh database for Django and importing those files into the new database. It may be a lot of extra work, but I think in the end it’s worth it. You will have a backup of your complete drupal site that you can store somewhere, and the new Django site will have only the bits you want.

 

Part 1. Setting up dev

 

I usually create the /sites folder first then each site-n folder is actually a Django app. So once you are ready issue the following command:

sudo djangoadmin.py startproject drupalimport

 

Then you can create the rest of your directory structure:

sudo mkdir scripts templates htdocs htdocs/media

 

Create a backup of your Drupal database so you’re not working with production data. I decided to create my backup inside a folder called drupal that was within my ‘drupalimport’ app. This folder will become an ‘app’ in order to bring it into Django.

sudo mkdir /sites/drupalimport/drupal

sudo chmod 777 /sites/drupalimport/drupal

sudo mysqldump drupaldb > drupaldb.sql -p

sudo touch drupal/__init__.py

* Alternatively you might also consider install the Backup-Migrate Drupal module and use the backup .sql file it generates

* If you don’t chmod 777 you will get permission denied when you attempt to write your sql file

 

You will want an interim database to use while you perform the migration. You will import your Dupal data into it and use it as your Django database as well. This will make it easy to copy the information from the Drupal tables to the Django tables that you have.

sudo mysqladmin create djangodb -p

 

Import the Drupal data into the newly created Django db:

sudo mysql djangodb < drupaldb.sql -p

 

You will want to setup your Django site to see the blog and admin apps at the very least. You will also want to enable the admin urls and blog urls so you can verify everything is working properly. In order to migrate the data from Drupal you need to sync the database so Django builds in the appropriate tables.

sudo python manage.py syncdb

 

You should now be ready to begin the data migration.

 

Part 2. Python Magic

 

Carson helped me work this all out, as I am in no way comfortable using Python. Django needs to model the database so that it can manage it.

sudo python manage.py inspectdb > drupal/models.py

 

This will create a models.py file that should contain a list of all the tables and fields. In our experience we noted that the Node table has a nid field which is a key field. In order to actually get this to work we had to modify a line in the models.py file. Open the file and search for the following string, “class Node” this will take you to where you need to be, and change the nid line from:

nid = models.IntegerField()

To:

nid = models.IntegerField(primary_key=True)

 

Save and close the file, now we can bust out the Python shell magic:

sudo Python manage.py shell

 

You will need to import the the Drupal model, the blog model and the date and time:

from drupal.models import *

from blog.models import *

import datetime

 

Next you need to create yourself an author:

author=Author()

author.name=”Jeff Patton”

author.email=”jeffrey@patton-tech.com”

author.ident=”jeff-patton”

author.save()

 

Set the site so the blog entries will show up:

site=Site.objects.all()[0]

* Don’t worry what this is you can change it later

 

I decided I wanted to tag all these entries so I could differentiate between the Drupal blog entries, Drupal book entries and so on:

tag=Tag()

tag.tag=”Drupalblog”

tag.save()

 

My example will show you how to pull the blog entries from the Drupal tables, you can use the same method to pull whatever information you need out. The two tables we are interested in appear to be Node and NodeRevisions. Node contains nid, title, and so on whereas NodeRevisions contains the actual body of the post/node. In order to pull them out we need to create an object of just blog entries and loop through it:

nodes=Node.objects.filter(type=”blog”)

 

That’s really all there is, you would substitute blog for whatever you were interested in, such as:

blog

geshinode

book

page

* These are the nodes I have, you may see different nodes and should looke at Node.type to see what’s available in your installation

 

Now that we our object setup we need to loop through it and pull out a particular blog article (nid) and store it as an entry:

for node in nodes:

entry=Entry()

entry.pub_date=datetime.datetime.now()

entry.slug=node.nid

entry.headline=node.title

entry.author=author

entry.abstract=NodeRevisions.objects.get(nid=node.nid).teaser

entry.content=NodeRevisions.objects.get(nid=node.nid).body

entry.save()

entry.sites.add(site)

entry.tags.add(tag)

 

Hit enter a couple times after this and it should go. Now for the fun part! Login to your admin site and you should see all the entries fully populated. I haven’t quite worked out the tags yet, but here are a few things you could do. First all the tags in Drupal are stored in Term_Data.Name, so you could just add them all in manually, or in a loop like above. If you want to be all fancy then here is the gist of what you need to understand.

 

There are basically three tables we are interested in; Node, Term_Node and Term_Data. Term_Node is a junction table between Node and Term_Data. There is a one-to-one between Node.nid and Term_Node.nid, there is also a one-to-one between Term_Node.tid and Term_Data.tid. My guess at this point would be that in place of entry.tags.add(tag), there would be a for loop akin to the following.

 

For each Term_Node.tid where Node.nid = Term_Node.nid entry.tags.add(Term_Data.name)

 

If you receive an error let me know I’ve run into a couple of issues:

Index out of range: Resulted in trying to add the site to the entry before the site entry was created (saved) which is why the save statement is where it’s at.

Permission Denied: When doing the ‘mysqldump’ into a folder that wasn’t chmod’d 777

 

Part 3. Setting up prod

 

Once you have migrated your data from Drupal into Django you may want to have a fresh database to work with. That is to say a database that has no Drupal information within it. The nice thing about this process is that the .sql file that you created earlier will serve as a backup for your Drupal site. If at some point you decide to go back to Drupal, it’s simply a matter of importing that .sql file into a Drupal database of the same name and specifying that during the Drupal installation.

We will use ‘mysqldump’ to export out the tables we’re interested in to a .sql file and then create our fresh database, import the sql file and point our new site at our clean database. The tables we’re interested in are below:

blog_author

blog_entry

blog_entry_sites

blog_entry_tags

blog_tag

 

Now create your backup:

sudo mysqldump djangodb blog_author -p > blog.sql

sudo mysqldump djangodb blog_entry -p >> blog.sql

sudo mysqldump djangodb blog_entry_sites -p >> blog.sql

sudo mysqldump djangodb blog_entry_tags -p >> blog.sql

sudo mysqldump djangodb blog_tag -p >> blog.sql

* See note in Part 1 regarding chmod

 

You now have the .sql file that you can insert into a fresh database:

sudo mysqladmin create djangosite -p

sudo mysqldump djangosite < blog.sql -p

sudo djangoadmin.py startproject djangosite

sudo mkdir scripts templates htdocs htdocs/media

 

Configure your site to point at your new database, configure the urls and update your Apache configuration and you should be good to go!

Attached is the import file that I created with Carson’s help. I had an issue where I didn’t have my Django variables setup properly for it to just run. So I opened a shell and pasted it there and it just worked. I’m sure I’ll have missed something, so let me know if there is a problem!

Windows Server 2008 R2 Core Net Server

I wanted to setup a DNS and DHCP server for my home network. I could have used a Linux server to do this, but I wanted to write about server core. I also wanted to mess with working over ssh to do things, I found a SourceForge project that worked really well on 2008 x86, I had errors on 2008 R2 which I’ll poke at next week.

I don’t think I need to explain these things too much, I’ll be using DNS and DHCP as they were intended to be used. I’m installing the SNMP service because I have setup a Zenoss server. It uses SNMP to monitor everything from disk usage to network usage. You will need to have your network card set to a static IP in the proper network for it work correctly.

PowerShell Installation

Windows 2008 Core

Install PowerShell following these instrcuctions.

Windows 2008 R2 Core

Install PowerShell following these instrcuctions.

Install OpenSSH

  • Download the install file
  • The installation takes care of everything but the firewall
  • Allow SSH through the firewall

    • netsh firewall set portopening TCP 22 “OpenSSH TCP22”

Basic Networking Configuration

netsh interface set interface name=”Local Area Connection” newname=”Public”

netsh interface ipv4 set address name=”Public” static 192.168.1.1 255.255.255.0 192.168.1.254

netsh interface ipv4 set dnsserver name=”Public” static 192.168.1.1 primary

netdom renamecomputer %computername% /newname:{newComputerName}

Installing the required services

DNS

start /w ocsetup DNS-Server-Core-Role

DHCP

start /w ocsetup DHCPServerCore

SNMP

start /w ocsetup SNMP-SC

Configuration of these services can be done via a server with RSAT installed, but that would be no fun at all, so we’ll use the builtin tools to set things up. If you have read the above article you may already have SSH setup and be using puTTY or some other client to perform these tasks.

Configuring the required services

DNS: dnscmd

The following command creates a forward lookup zone and marks it as the primary zone on this server.

 dnscmd ServerName /ZoneAdd company.com /Primary /file company.com.dns

The following command creates a reverse lookup zone for 192.168.1.0/24 and marks it as the primary zone

dnscmd ServerName /ZoneAdd 1.168.192.in-addr.arpa /Primary /file 1.168.192.in-addr.arpa.dns

The following command creates an A record for the local server at 192.168.1.1

dnscmd ServerName /RecordAdd company.com @ A 192.168.1.1

The following command creates the reverse record for the server at 192.168.1.1

dnscmd ServerName /RecordAdd 1.168.192.in-addr.arpa 1 PTR company.com

Allow DNS traffic to pass through the firewall

netsh firewall set portopening TCP 53 “DNS-TCP”

netsh firewall set portopening UDP 53 “DNS-UDP”

DNSCMD Resources:

Setup DNS from the command-line

Configuring DNS on Server Core

DNSCMD Cheat Sheet

Administering DNS Server

DHCP: netsh

After installing the DHCP role on the core server you will need to run the following command to allow the service to auto-start

sc config dhcpserver start= auto

 

Production Script: UpdateADDescription

The UpdateADDescription is run hourly on a computer connected to the domain. It’s sole purpose is to loop through all the computers in a given OU and update their Description property. It serves as both inventory and somewhat of a user tracking system as it will give us an idea of who is logged on to what computer.

Most of the functions and procedures in this script have already been documented, so to get the details on how everything interacts you might look to those articles first.

I wanted to be able to use the Active Directory to hold information that was pertinent to how we deliver service to our users. I needed something that would let me know who was logged on, where they were logged and some inventory-ish data about the computer itself.

The beauty part about this is the output of the script is stored in a comma-separated format in the Description property, so in the Active Directory Users and Computers console you can literally export the view as a comma-separated file, and open it directly in any spreadsheet program.

The main fields from the view are comma-separated and the Description property, comma-separated is parsed out like additional fields.

In order to use this script in your environment I highly suggest you run it in a test environment.

DPM 2007, Windows 2008 R2 and Windows 2008 Core Domain Controllers

Still bringing services online after our recent hardware upgrade, today’s goal was to make DPM 2007 go. The original server was a PoiwerEdge 1950 with a MD1000 attached. It was populated with 15 750gb drives and housed all of our backup data. As both the server and drive array were falling off warranty we took this opportunity to purchase a new PowerEdge server, but we opted for a PromiseArray with 16 2TB drives.

DPM Server is running Windows 20008 R2, the OS installation went fine, there are several tutorials out there, I followed the one from the Core Team. There is one thing that you will need to change though in order to get reporting to work properly.

Error Message:

DPM could not connect to SQL Server Reporting Services server because of IIS connectivity issues.

On the computer on which the DPM database was created, restart the World Wide Web Publishing Service. On the Administrative Tools menu, select Services, Right-click World Wide Web Publishing Service, and then click Start.

ID: 3013.

This message is due to the new security features, I found this article extermely helpful for 2008 R2.

http://technet.microsoft.com/en-us/magazine/dd673659.aspx
start /w ocsetup WindowsServerBackup

http://technet.microsoft.com/en-us/magazine/dd630943.aspx
Wmic product get name /value
Wmic product where name=“Name” call uninstall

http://social.technet.microsoft.com/Forums/en/winservercore/thread/5a438757-d294-483d-8619-df9eb5700561
netsh firewall set opmode disable
netsh firewall set opmode enable

http://technet.microsoft.com/en-us/library/cc161275.aspx
netsh firewall set portopening TCP 3148 “DPM TCP 3148” ENABLE
netsh firewall set portopening TCP 3149 “DPM TCP 3149” ENABLE
netsh firewall set portopening TCP 135 “DCOM TCP 135” ENABLE
netsh firewall set portopening UDP 137 “NetBIOS UDP 137” ENABLE
netsh firewall set portopening UDP 138 “NetBIOS UDP 138” ENABLE
netsh firewall set portopening TCP 139 “NetBIOS TCP 139” ENABLE

netsh firewall reset
netsh firewall set service RemoteDesktop enable
netsh advfirewall firewall set rule group=”remote administration” new enable=yes
netsh advfirewall firewall set rule group=”Windows Firewall Remote Management” new enable=yes
netsh firewall set logging filelocation=%systemroot%system32LogFilesFirewallpfirewall.log maxfilesize=4096 droppedpackets=Enable Connections=Enable