Tuesday, May 26, 2009

SQL Login Hell

No apologies for my thinly disguised reference to DLL hell in the post title, I think everyone dealing with SQL Server has at one point dealt with login issues (e.g. two colleagues today, hence the post). If you're like me, you'll see these issues again and again and each time you'll have to Google them to remind yourself how to solve them. I'll add to this list over time but for now, here's a couple of regulars:

Error: Login failed for user 'sa'. The user is not associated with a trusted SQL Server connection.
This just means that you haven't set the Server to use Mixed Mode authentication. Login using Windows authentication and change your server authentication settings on your server to look like this:

Finally, restart the database engine's service. The easiest way to do this is through Control Panel > Admin Tools > Services. Click on SQL Server (Instance Name) and restart the service. You should then be able to login to the server with the sa account without a problem.

Error: Login failed for user 'sa'. (Without any other message)
Check that:
  • - the sa account's password is correct or that it is strong (if enforce password policy is checked)
  • - the account is granted permission to connect to the server
  • - the account is enabled
  • - the account is not locked
Error: Cannot open user default database. Login failed. Login failed for user 'user.name'.
I logged in with Windows authentication and dropped the database that was set as the default database for this authentication. I also forgot the sa account's password so I couldn't log in with that to change my Windows authentication's default password. That leaves scripts! Using the master database's context, execute the following command (replacing the names where appropriate):
A little special Olympics, but I managed to do this. Or perhaps that just says something about me...

Sunday, May 24, 2009

Some SSIS pointers

Here's a few pointers that'll make your life with SQL Server Integration Services (SSIS) a bit easier:

Set your database table fields to Unicode character types
If your data sources come from MS Excel and Access, and you're not using Unicode character types, you'll need to include the Data Conversion data flow task. It's pure grunt work that you'll need to include for each table you're migrating, and it can be easily avoided.

Avoid repository conflicts by working on separate packages and merging later
Using SVN, this has been a nightmare. Sometimes your packages merge perfectly, other times you have many lines of conflict that are just not worth the effort of resolving. It's easier in that situation to copy and paste the work you've created to another package, delete the file and get it back from SVN again, re-add your files and commit.

As a temporary stop measure work on separate packages, within the same solution, from your colleagues and designate one person to look after a "main" package (essentially manually merging your work). This also prevents an irate colleague from walking over to your desk several times a day when you keep breaking their connection strings and giving them gifts of 29 errors and 209 warnings that they can only resolve through re-establishing the connections in the each of the data flow tasks.

Only use a SQL Server Destination if your packages are going to run on a local server
Use an OLE DB Destination if your SQL Server instance is located on a remote server. An issue with this is that you don't get all the functionality associated with SQL Server Destination (and, most importantly, the speed!). A quick MSDN lookup will show you that this is in fact a feature, not a bug! Such is life...

Saturday, May 9, 2009

Don't forget the data!

Data migration is a nightmare. You never have enough time for it, and the source of the data always looks like this:

And I've got that expression on too. You're likely to encounter this situation at some point, unless you have the luck of writing a brand new system that just requires basic master data (where do I sign up?!).

I've been happily impressed though, with SQL Server Integration Services (SSIS). It's commonly used to do basic mapping and importing of data from Excel from within SQL Server, however using Business Intelligence Studio, you can create some powerful data migration packages by using a workflow of data transformation tasks. Very 5th generation, says a colleague. I'm not going to give a huge breakdown, as that is more effectively given by the SSIS project on CodePlex, but here's a basic example of a data flow task:

The greater Control Flow can contain many data flow tasks as well as other tasks (eg Bulk Inserts, Execute SQL, FTP Tasks, Sending Mail and many more). The data flow is probably the most complex, as this is where your specific data transformations occur. In the screenshot, you see that we read from a source destination (in this case an MS Access database), from which we do some data conversions, followed by a lookup of another field based on one in the source database (think of it like a VLOOKUP in Excel), creating a derived column, perhaps a calculation, and then inserting the transformed data into our SQL Server destination. Of course, you need to set up all of these by double clicking on the individual tasks and performing some MacGuyver tricks but you're hopefully getting the idea of how useful this tool is.

The key is that these packages are reusable. I've spent some many hours massaging the same data for different time periods in separate Excel spreadsheets. One of these packages could've got my public holidays back!

Testing the packages

While the proverbial jury may still be out on unit testing, I've personally found it invaluable in projects. Not as much during initial development, but its usefulness becomes reallyclear during support.

Now I've recently come across the ssisUnit project on CodePlex. This uses the familiar ~unit test format - setup, test, teardown (with setup and teardown used to create and remove test data, respectively) - using XML. I'm busy playing around with this and will post my thoughts up in the next week.

Now, providing a business case for the additional effort required to include any type of unit testing an a project is a whole other story...

Saturday, May 2, 2009

All it takes is a little thought...

I'd like to start off my blog by looking at a variation of a theme on a post by a friend and colleague on his blog regarding South Africa's Independent Electoral Commission's website.

A particular interest of mine, and one that is notoriously difficult for developers to get right, is the area of human-computer interaction (HCI). This seems to have been something the development team at the IEC have forgotten about at best, or ignored at worst. After the recent South African General Elections, I thought it would be interesting to have a look at the breakdown of the voting patterns in the area I live in. I opened up the report page to find this handy breakdown:

Now granted that whilst the Voting District number is supplied in your identity document, would it really have taken much trouble to populate this drop down with the name of the Voting District, as they've done for the Province and Municipality? This is a standard ASP .NET control, quite simple to populate the name and value properties...

Ah, but never mind - there is a handy search tab I see. Well, not quite. This allows you to search the Voting District's number, or by some other variable called the "VS Name". On running a report you discover this stands for "Voting Station Name" - as in the physical building you voted in (e.g. "Camps Bay Club"). I tried searching for large, well known suburbs but nothing turns up. I tried an exact search of Camps Bay Club but nothing turns up. Clearly this piece of functionality is quite flawed. I was nearly tempted to call them on the toll free number for that "more information", seeing as there was no discernible help available!

This is could be all quite innocent - until we view the source. I must say, they've out done themselves with unnecessary Javascript. I don't want to stray too far off the HCI topic, but even at Varsity I knew I was doing something wrong if I had to use invisible text fields to get the job done. And why write lines of Javascript for something that can be easily achieved using the .NET Framework? In fact, why bother using the framework at all?

Finally, I had a look at the validation. Take a look at this function,
presumably used to prevent HTML injections (taken from the view source available in any browser):

This function is used to detect the characters '<' and '>' when the user enteres them
onto the screen or when text containing the mentioned characters is entered onto the screen.
When this happens, this function will delete remove everything that was entered onto the textBox
function RemoveBadCharacters()
var txtSearch = document.getElementById('ctl00_ContentPlaceHolder1_txtSearch');
var bRemoveText = "0";
if(window.event.keyCode == 188 || window.event.keyCode == 190)
txtSearch.value = "";
for(var i = 0; i <= txtSearch.value.length; i++) { if(txtSearch.value.substring(i, 1) == ">" || txtSearch.value.substring(i, 1) == "<") txtSearch.value = ""; } return false; }

Whilst it's great that they've thought about validation it's not so great that they don't tell the user what they've done wrong. Rather let them enter these characters, and redirect an error message to the page. And see what I mean with regards to unnecessary Javascript? Not much room for code reuse across the application here. Again, I'm straying.

Now I've knocked this application quite a bit, so I need to be fair as I'm not aware of their requirements, deadlines, development team skills and all the other holistic aspects that affect development. However, this is an important institution, reporting on an event that only occurs once every 5 years - and this reporting was done far more innovatively by others (News24 is a great example). I think they're some key principals we can take away from this:

  • Apply the KISS (Keep It Simple, Stupid) mantra - to both the usability as well as the code
  • Make it clear to the user what they've done wrong, and how they can go about correcting it
  • Make it easy for the user to do the right thing, and difficult to go wrong
  • Provide easily accessible, preferably context based, help