Most of my attention this week was occupied with following Mix 2010. So, it shouldn’t be much surprise that a lot of these bullets stem from that.

• BIDS Integration Story in R2 – The Good, the Bad, and the Ugly
  “There was a huge discussion thread about the BIDS-Visual Studio integration story in the SQL Server 2008 timeframe where customers complained that BIDS got married with Visual Studio. This required switching both BI and code projects at the same if you want to have a solution with both project types. What has changed in R2?” – [Prologika]
• Programming Windows Phone 7 Series (DRAFT Preview)
  “A reboot is what Microsoft has initiated with its new approach to the mobile phone market. On February 15, 2010, at the Mobile World Congress in Barcelona, Microsoft CEO Steve Ballmer unveiled the Microsoft Windows Phone 7 Series and promised a product introduction in time for year-end holiday shopping. With its clean look, striking fonts, and new organizational paradigms, Windows Phone 7 Series not only represents a break with the Windows Mobile past but also differentiates itself from other smartphones currently in the market.” – [Microsoft Press]
• Windows Phone 7 Series hardware specifications revealed!
  “Microsoft today at Mix officially revealed the hardware specifications of Windows Phone 7 Series. So, now all the manufacturers have to follow these minimum set of specifications for every Windows Phone 7 Series device.” – [Microsoft Feed]
• Silverlight for Windows Phone
  “Silverlight for Windows Phone supports core Silverlight capabilities in managed .NET code with XAML…” – [Silverlight.net]
• The Right MIX
  “Today at MIX ’10 we’re looking under the hood of Windows Phone 7 Series, showing people what this phone can do and giving designers and developers the tools and information they need to expand their horizons.” – [Windows Phone Dev Blog]
• Introducing Expression Blend for Windows Phone
  “Windows Phone is an amazing platform to use and to design applications for. We are very proud to make a preview version of authoring for Windows Phone available with Expression Blend 4.” – [electric beach]
• Developing for Windows Phone 7 Series
  “Some time back the team and I talked about our commitment to the Silverlight and XNA platforms on Windows Phone 7 Series as the primary developer platforms.” – [artificial ignorance]
• ORAYLIS BI.Quality
  “ORAYLIS BI.Quality is a testing suite for BI solutions makes it easier to develop in an agile environment. The suite is based on NUnit and supports quite a lot of different testing methods.” – [CodePlex]

Ok, that was very phone heavy. What can I say? I am excited about the new platform. There are several videos posted on the official Mix site for many of the other keynotes and sessions. I recommend that you go take a look.

Traditionally, as stated in the previous post, when Microsoft is referring to the three-screen concept they are talking about PC, mobile, and TV. The actual devices behind each of these can vary. In fact I would go so far to say that the devices themselves are not the important component. How a user interacts with the device and how the device is connected to the cloud ultimately should determine how it is categorized within the concept. So, let me start with the PC and what that screen represent (at least in my opinion).

When you’re targeting the PC as one of the “screens” it is common to think of it as a computer with a monitor, keyboard, and mouse. If, however, you look at it as a primarily stationary device that is always connected and where users actively interact with it in close proximity of the display it starts to open up to more devices beyond the classic desktop PC. So, what kind of devices are we talking about?

How about a multi-touch PC mounted in a kitchen? A user walks up to it and interacts directly with the screen as opposed to a keyboard and mouse. But, the interaction takes place within the same general proximity as a traditional desktop. So, from a display perspective it needs to be optimized for close viewing.

An end-user is not going to sit down and type a novel on this device. Instead, they would use it to display recipes, maintain a family calendar, or watch video with a few taps of the finger. Yet, this is essentially still a PC and I do not think many would argue against it falling into the PC screen category. But, it does start the move away from what is thought of as the typical PC. And, I hope it starts you thinking about other possibilities as well. So, let us move a little bit further from the traditional PC again. How about a Microsoft Surface device?

Most of the Surface devices that have shown up in the wild have been in hotel lobbies, bars, and retail stores. And, for the most part they have been little more than a novelty. But, there is reason to believe that there can be more to this than what we have seen so far.

Several months ago I watched a video (of course I can’t seem to find a link to it now) where Microsoft demoed a Surface desk that was tailored to a factory supervisor. The interface itself was more like a sophisticated dashboard than traditional GUI and was fully multi-touch. On this dashboard the supervisor was able to see at a glance a schematic of the factory floor. Individual equipment on the schematic was color coded to indicate if there was a problem. For example, maybe a line was not meeting the production quota that is has been assigned. Or, perhaps maintenance had been recently skipped on the machinery. With a few taps the supervisor is able to see historical performance information as well as future data such as employee shift schedules.

The demo continued by showing additional cool features of the Surface device. For instance, the supervisor was able to perform and enterprise search for an electronic document by simply placing the hardcopy face down on the desk surface. All of this is very interesting and shows some real value.

Ultimately, the interface was seamless, customizable, and geared directly towards complementing the supervisor’s natural workflow. Here again, we have many of the same concepts of the traditional PC. But, it is really starting to open up some exciting possibilities; especially for BI. Certainly there are some big differences between this and the display that sits on your desk. Yet, I’d contend that this is still the same “screen”.

Microsoft has been touting their technology vision of three-screens and the cloud for about a year now (much longer if you don’t include the formal name). For those not familiar with the concept it is essentially a vision that embraces the union of technologies across the PC, mobile, and TV. The common connector in this vision is usually the cloud.

Over the last several months the bits and pieces that Microsoft has shown of this vision have been fragmented and lacked a sense of completion. However, with the recent announcement of the Windows Phone 7 things are starting to come together; at least on the consumer side of things.

This week the GDC (Game Developers Conference) is taking place and Microsoft is showing off the gaming portion of their vision with announcing XNA Game Studio 4. One of the demos that they recently presented showed a game that worked seamlessly across the PC, Xbox 360, and Windows Phone. Even more impressive than simply having a game that can be ported across these platforms is that the game was able to push it’s save state to the Cloud. This allows the end user to pick up where they left off across all of their devices.

This is certainly cool and is exactly the type of thing that gets my imagination going. I innately have a curiosity for all things dealing with technology. And, when I see something like this my mind immediately starts racing with different scenarios of how this can be used.

With that in mind, I had originally intended to take this post and talk about how this technology vision could be applied to business intelligence; more specifically, how Microsoft intends for it to apply to their product lines. But, after some additional thinking I decided that I wanted to take another direction. Instead, I’d talk at a more abstract level. I’d like to bring up some ideas on how to expand this vision and perhaps redefine it a bit. The idea behind this is to allow your curiosity and imagination to run wild as opposed to having me just dictating to you. To me this is a far more powerful prospect.

With this change in direction the amount of content has grown larger than what I’d consider appropriate for a single blog post. So, I am going to break this up into a small series. It is my hope that you will find this not only interesting but that it will prompt you to imagine and, perhaps, help create the future!

While catching up on blog posts I came across a post by Patrick Husting from last month. The post titled “Better looking charts in Excel 2007/2010” provides a before and after example of a chart in Excel. The simple example is meant to show the importance of adding a little style to your reports.

I think this is an often overlooked aspect of a complete BI initiative. Too often the focus is only on the technical aspects and does not ultimately address user adoption. While there are several things that can contribute to low user adoption having ugly and unintuitive reports doesn’t help.

With this in mind, I would argue that taking the time and effort to design a comprehensive style guide is well worth the investment. If done properly it should help a project realize ROI. So, when you’re planning your next project consider budgeting time for style planning.

Anyhow, check out Patrick’s blog and let me know what you think.

This is a continuation on my previous rants on borrowing traditional software development concepts for a BI project. As the title suggests, the topic today is source control.

No matter what you want to call it, a cornerstone of any good software project should be source control. This is pretty fundamental for software projects but it hasn’t been a priority for many in BI. I am not entirely sure why this is. Is it just an oversight? Are BI project managers oblivious to the existence of good source control tools? I’m not sure what the answer is but I am sure that BI teams should be using it.

The benefits of source control should be well known. So, I am not going to rehash them here. However, I would like to go over a few things to keep in mind:

Full Coverage

Source control can be applied to most parts of a BI project. Source control concepts can apply to data models, cube structures, ETL, and even the documentation for the project. Pretty much anything that changes over time can benefit. So, try to think beyond just source code.

Version Labels

Having the ability to identify something makes it infinitely easier to discus it. This same concept applies to components of a project. It doesn’t really matter whether it is some sort of name or just a revision number. The benefit is that it allows you to have a common term to refer to a snapshot of a project.

I strongly suggest that the labeling is universal and span all of the projects components. In other words, version 2.1 could universally refer to a snapshot of the ETL as it does to the data model. And, when you start introducing formal issue tracking and end-user team portals it becomes even more important (more on these topics at a later date).

Ultimately what I am getting here is that checking in revisions isn’t enough. Develop a consistent labeling system for releases. And, use the commenting features of your source control system to identify changes; all changes!

On Cost

There are many source control systems out there. And, many of these are free. So, cost should not be an excuse. Additionally, many of the design tools on the market today have some flavor integrated into the development environment. Don’t make excuses, find something that works for your team and use it!

Previously, I had discussed treating Business Intelligence projects in a similar manner to software development. Continuing on this subject another often overlooked item in BI projects is release management.

So, what do I mean by release management? Release management is a large and ever evolving topic in software development and often speaks to the entirety of an IT organization as opposed to a single project. However, for the purpose of this post I am really only interested in one thing: protecting the live production environment.

I have had numerous clients who seem to believe it is alright to develop directly against a live production environment. Yet, more often than not, if you walk down the hall in these same organizations you’ll see that their custom development teams follow strict guidelines for deploying new code to production. Why is this?

Often times these organizations look at BI as being non-critical. If a new release results in unexpected downtime it is viewed to be not as important as the uptime of a transactional business system. And, in an immature BI environment this is probably true. However, if an organization wants to realize the ROI needed to justify continuation of a BI program they are going to want to reach maturity.

Holding back on a sound release management plan or treating a BI project like a second-class citizen is only going to work to slow this maturity. It will hurt the overall perception of the program and will result in poor user adoption. If a user cannot count on a system to be fully tested and available why would they trust it? Ultimately, I think you will find it is far cheaper to start these processes as early as possible.

Fortunately, it is pretty easy to get going. Start with the environments. Have a formal development, testing, and production environment. This doesn’t need to be difficult or costly. The development and test environments do not need to be as robust as production. They just need to be able to isolate the changes from production. In most cases you’ll be able to use a subset of the overall data for development and testing purposes.

Next, establish some guidelines for promoting changes to the individual environments. If you’re using one of the agile methodologies for BI development it most likely already has an established process for this that you can follow. Overall, the governance behind this process can evolve and expand with the BI program.

Finally, the results should be a release process that allows developers to continue enhancements of the system, testers to verify the quality of the changes, and the end users rely on the availability and accuracy of production. I firmly believe that this is an essential part of any successful BI project. Can you proceed without it? Yes, but you’re going to pay for it in the long run.

There are many ways that a BI project can fail. And, many do just that. In my experience one of the top reasons is poor project management. Opinions as to why seem to fall into two camps. One side believes that BI is not inherently an IT project but is instead a business project and treating it like an IT project leads to failure. The other side opines that years of project management principles have been refined for software development and BI should learn from this.

Personally, I feel that both sides have their merits. I fully understand that there are some important differences with a BI project. And, there is no doubt that it must be driven by the business. Yet, I lean towards treating BI projects in a similar manner to software development project. In the future I hope to go into detail on how BI projects are different and hopefully will be able to provide advice on how to effectively manage these differences. In the meantime, what does it mean to treat a BI project like software and what exactly should be borrowed from the software development world?

Be Agile

To start with, the scope of a data warehouse and the ultimate business rules and requirements that it will encompass are vast. I contend that it is not possible to distill all of the business knowledge that is needed into a scope document; BI projects that try are bound to fail (at the very least they are susceptible to huge cost overruns and user adoption issues). With that thought in mind, a BI project does not lend itself to a traditional waterfall model.

Instead, a BI project should be agile and an iterative approach should be taken. There are several flavors of agile development methodologies and they are typically not geared towards a non-software project. So, lets look at some of the things that they do have in common: iterative development, a focus on team collaboration, and, perhaps most important, the ability to be adaptive to project change.

An Iterative Approach

It seems that many BI projects focus only on the big picture. The project can be on the small side and tasked with expanding a single subject area in an already existing warehouse. Or, it can be a huge build of an enterprise-wide data warehouse. Regardless, the focus often is on the pristine and perfect end product. I would argue that this is an enormous mistake.

There are a few reasons why I believe that this is a mistake. But, let me focus on the one. This style of development lacks transparency and creates a black box. It is difficult for the business to see the value in what is being developed. Instead, what they see is a lot of man hours being spent and they can quickly lose confidence in the project. This can be the death nail for a project and may very well hurt future BI efforts as well.

Alternatively, most BI projects can be broken into small components that take a short duration to complete. Each of these components can be prioritized and assigned to team members for a given iteration. Each successive iteration expands on the previous. So, after a single iteration you may not have a complete deliverable product but you do have something to show.

Take this example: you have a proposed large many-sourced product dimension that needs to be conformed. The work required to accomplish this task is greater than a typical development iteration. However, it can be broken down into even more fundamental parts. For instance, you might target a prioritized subset of the dimension attributes.

In the first iteration you’re able to have team members perform the business analysis work required to assess data quality, build a basic data model, and create the ETL to populate this data structure. In the end, you are not able to deliver a complete product. But, you are able to demonstrate progress. And, in each successive iteration you are able to build on this progress, learn from the work of the previous iteration, and ultimately deliver a complete conformed product dimension.

It seems simple enough. And, with a good team, it is. Each step of the way the business stakeholders have insight into what is being done. And, the likelihood of success is greater.

Still, there does needs to be some caution. A BI project is really just a small piece of the over all pie and should be part of a larger BI program within the organization. It is important to have the big picture in mind while working through the individual parts. You need to be conscious of and avoid building silos of data. The goal still needs to be to have a comprehensive and conformed view of the organization. It isn’t that agile doesn’t allow for this just be careful that goals do not get lost during the daily focus of an iteration.

Team Collaboration

Working with a team certainly isn’t something that is unique to agile development. But, there does seem to be a particular focus that shouldn’t be ignored. An effort should be made to have face-to-face interactions between the team members and it should be routine. Any roadblocks, as well as progress, should be brought to the teams attention as quickly as possible. And, attempts to involve business stakeholders should be a priority. After all, a BI project is for the benefit of the business and not just a pet project of IT, right?

Once again, much of this is about transparency. But, it is also about effectively working together and addressing problem early.

An entire book could be written on this subject and I can do it little justice in the limited space I have here. I simply encourage you to research the team methods used by the various agile flavors. And, please, do not overlook the importance of this. Good teamwork can make or break a project.

Adapt

A plague to many BI project (and software projects as well) are rapidly changing requirements. Many times the source of this is simply the result of a project digging deeper and deeper into the dark inner workings of a long running organization. Or, it could be the stakeholders changing their minds mid-stream. Whatever it is, it happens, it isn’t predicable, and you need to be prepared to deal with it.

The short iterative agile cycles are able to easily adapt to this change. While you will still have a high level project plan, required features, goals, and priorities the only thing that is really set in stone at any given moment is the current iteration. If a spec changes it is simply added to the list for future iteration developments. It is no longer an unforeseen showstopper. It is now just another requirement in the overall pool. In many ways this is a very liberating concept. Of course, there are still consequences to scope-creep. But, it is now an integral part of the process as opposed to a wall stopping progress.

Now What?

It goes without saying that agile methodologies are not perfect and that they will not solve all of the problems that a BI project will face. They are, however, flexible and able to provide a good base to build upon. As a part of a larger strategy they can provide important direction and proven guidelines. So, should you choose a specific agile methodology for your BI project?

As stated above, typically they are geared specifically towards software development. The flexibility, however, should allow you to adapt a methodology to a business intelligence focus. In fact, as an IT strategy it may be very possible to use a single methods across multiple project types (Scrum comes to mind). But, agile development methodologies are more about a mindset than a specific series of techniques. So, do not spend too much time dwelling on what some software pundits have turned into a religion.

The bottom line is that if you choose, and you should, to use a more agile development methodology in your BI projects you are going to need to adapt and tailor it to your organization. It is a mistake, however, to dismiss it as not being important or just the latest fad. Keep in mind that this is just a start. I will continue to expand on BI project management in future posts. If you’re not a believer now, just give me some time.