Just about every developer out there has encountered a scenario where they need to load test the application they are creating, but simply cannot come up with enough load to make it happen. I'm not going to go down the rabbit hole that is load testing an application in this post, but I do want to talk about one component of load testing, which is working with a lot of data. I seem to always run into the situation where I'm developing an application for a client, and when it comes time to test the application in a real world scenario and need real world data, the only viable solution is to back up data from a production server and massage it to "fit" into my applications database. Bad idea. Yes, there are times when this is unavoidable, but in general, do you really want to make multiple copies of sensitive data? Also, in most cases I cannot just copy and paste the data into my database. In the past I've created a number of data migration utilities, data population scripts, and used a number of tools just to populate my application with data that I can use to test it. All of this takes time, and effort.
In contrast, there is Red Gate's SQL Data Generator. The SQL Data Generator is aptly named, because that is what it does, it generates data. And good data in fact. I downloaded the product trial recently when trying to hand jam a data population script for over 1,000,000 rows of data in multiple keyed tables. Needless to say, I became a little frustrated and started looking for alternatives. I downloaded the tool to my machine, installed it, and generated 1,000,000 rows of data for all of my tables in under 10 minutes. Nice. In addition to simply generating data, the SQL Data Generator is smart enough to create records that enforce referential integrity, that match data types, and in some cases, even provide data reflective of the column name. In one case, I have a column that is of type VARCHAR named STATE. The data created included all 50 US State names.
Wednesday, November 30, 2011
Wednesday, October 19, 2011
SSIS Attunity Connectors
Recently I had to create an SSIS package that transferred a huge amount of data between an Oracle database and a MS SQL Server database. I began creating an SSIS package that used an existing linked server connection that the client had already been using to perform similar data transfers. To my dismay, the performance of the linked server was dismal. Instead of using the linked server I decided to go with the OOTB OLE DB data source available in SSIS. This increased my performance dramatically. However, when I spoke to a colleague at TCSC, he recommended that I try the Microsoft Connectors for Oracle by Attunity. I had never heard of these connectors, but I was intrigued, so I downloaded them and ran some performance test.
The following tests results we recorded while debugging through Visual Studio, so real numbers will be better than what I have recorded, but the results do represent without a doubt the performance differences between the different connections.
The following tests results we recorded while debugging through Visual Studio, so real numbers will be better than what I have recorded, but the results do represent without a doubt the performance differences between the different connections.
Attunity
|
OLE DB
|
Linked Server
|
|
3000 rows |
2 seconds
|
6 seconds
|
10 seconds
|
10,000 rows |
3 seconds
|
10 seconds
|
20 seconds
|
100,000 rows |
19 seconds
|
1 minute 28 seconds
|
3 minutes 2 seconds
|
500,000 rows |
1 minute 23 seconds
|
7 minutes 35 seconds
|
16 minutes 18 seconds
|
1,000,000 rows |
2 minutes 38 seconds
|
16 minutes 37 seconds
|
30 minutes 28 seconds
|
Saturday, July 16, 2011
Upgrade your version of WSPBuilder for a more traditional development experience
In many ways a seasoned ASP.Net developer can have a difficult time learning SharePoint 2007 development because they have become accustomed to the practices ascribed to by the Microsoft ASP.Net team. One of these practices is so benign and intuitive, that most developers find it incomprehensible that it is not available when using a WSPBuilder project template. The practice that I am referring to is right clicking your declarative markup and selecting “View Code”. The menu item is not available, because the WSPBuilder Project type is based on the standard C# project type, and does not include the web application project type. Because of this, you cannot add regular web application based items such as web forms and web controls. This has lead many developers, including myself, to develop SharePoint solutions by hand, manually linking our declarative markup and code behind files, all the while not using designers and in some cases intellisense.
Friday, June 24, 2011
Visual Studio Styles
I was recently asked where I got the Visual Studio color scheme that I used in this post:
http://www.viamacchina.com/2011/06/a-better-inputformsection-for-sharepoint.html
Originally I had been turned on to the VS Theming trend by Scott Hanselman’s post about theming Visual Studio.
http://www.hanselman.com/blog/VisualStudioProgrammerThemesGallery.aspx
When I went back to his blog to get the link for his post, I saw that it has been updated and now there is an entire site dedicated to Visual Studio themes:
http://studiostyl.es/
There are a ton of great themes on this site, and you can even make your own.
For the most part theming your environment is just a fun way to beak up the monotony of an application that you spend WAY too much time with. But, I have also found that after switching to a darker theme that I have reduced the amount of eye strain and fatigue associated with marathon coding sessions.
Have fun!
http://www.viamacchina.com/2011/06/a-better-inputformsection-for-sharepoint.html
Originally I had been turned on to the VS Theming trend by Scott Hanselman’s post about theming Visual Studio.
http://www.hanselman.com/blog/VisualStudioProgrammerThemesGallery.aspx
When I went back to his blog to get the link for his post, I saw that it has been updated and now there is an entire site dedicated to Visual Studio themes:
http://studiostyl.es/
There are a ton of great themes on this site, and you can even make your own.
For the most part theming your environment is just a fun way to beak up the monotony of an application that you spend WAY too much time with. But, I have also found that after switching to a darker theme that I have reduced the amount of eye strain and fatigue associated with marathon coding sessions.
Have fun!
Saturday, June 18, 2011
A better InputFormSection for SharePoint
One of my biggest annoyances with developing solutions SharePoint 2007 is the fact that many of the web controls available in the SharePoint framework do not support the same development paradigm that traditional ASP.Net controls support. Specifically, many controls in the Microsoft.SharePoint.WebControls namespace do not support two way data binding. The best example of this is the InputFormSection control. Like many ASP.Net Templated controls, the InputFormSection control allows you to encapsulate items of your choosing in regions within the control. The control takes care of the layout and positioning of your controls, and reduces the need for redundant declarative markup.
Friday, June 17, 2011
GhostDoc - My favorite “New to Me” extension for Visual Studio
Commenting our code is a necessary but sometimes painful exercise for developers. We tend to scrutinize our object structure, method names, and overall terseness and readability of the code itself, but leave commenting for last. I recently ran across a free extension for Visual Studio that makes adding XML documentation comments for your code as simple as a key stroke.
Monday, June 6, 2011
Using a base web part class when developing SharePoint web parts
When developing solutions for SharePoint 2007, I use WSPBuilder almost exclusively. Carsten Keutmann has created an excellent tool that automates and eases much of the work that is involved in creating a solution for SharePoint. In fact, in some ways, WSPBuilder makes things too easy. When creating a project using the out of the box WSPBuilder project and item templates, it is easy to fall into some bad practices.
One of these practices that I see far too often involves the use of the OOTB Web Part Feature Template. If you add a new Web Part Feature to your project using the WSPBuilder Item Template, WSPBuilder will automatically add a Template folder, a Features folder, and then a folder of the same name as your web part. Within your web part feature folder, four files will automatically be added. An elements.xml file, a feature.xml file, a .webpart file, and your web part code behind .cs file.
One of these practices that I see far too often involves the use of the OOTB Web Part Feature Template. If you add a new Web Part Feature to your project using the WSPBuilder Item Template, WSPBuilder will automatically add a Template folder, a Features folder, and then a folder of the same name as your web part. Within your web part feature folder, four files will automatically be added. An elements.xml file, a feature.xml file, a .webpart file, and your web part code behind .cs file.
Thursday, January 13, 2011
Making use of the SPDisposeCheck utility
Most seasoned SharePoint developers have found out one way or another that you have to be mindful of properly disposing your SharePoint objects. Unfortunately, the rules, guidelines and general practices are often confusing and un-intuitive. Because of this it is common to find SPSite, and SPWeb objects that are not properly disposed. To help combat this issue Microsoft has released a tool called the SPDisposeCheck utility that will analyze your assemblies and report on any improperly disposed Shared Point objects.
The utility can be downloaded here:
http://code.msdn.microsoft.com/SPDisposeCheck
To integrate the utility into your development environment, in Visual Studio go to the "Tools" menu and select the "External Tools" menu item. Select "Add" and configure the tools options as follows:
The utility can be downloaded here:
http://code.msdn.microsoft.com/SPDisposeCheck
To integrate the utility into your development environment, in Visual Studio go to the "Tools" menu and select the "External Tools" menu item. Select "Add" and configure the tools options as follows:
Subscribe to:
Posts (Atom)