Wednesday, June 8, 2016

Continuous Integration Visual Studio SSDT with Conditional Output

I am often asked how do you run your database builds from Visual Studio with SSDT?  Often times there is a requirement to use MSBuild to complete the task of an integrated build solution and you want to perform different actions based on different project configurations.  For example you might want to run some code table load scripts after the database is created or you need to move data from an old table to a new table or set of tables.  To accomplish this you need to edit your SSDT project to enable different file contents to be copied and executed during the build and deployment of your project. Once the project is built the artifacts can be deployed to various servers or packaged for later deployment as part of a software package.  All of this can be accomplished with Visual Studio and SSDT along with MSBuild and some kind of build orchestrator such as Jenkins.

The goal of this article is to provide the framework within Visual Studio and SSDT to generate the desired output.  Within Visual Studio SSDT the tool leverages a publishing model.  This means that the build and deployment of the SSDT solution is run by clicking on the publishing action within a project or solution.  But what if you want a build server to produce the artifacts for deploying your solution to a development server or as part of a deployment package to be installed at a later date?  The idea is simple however getting it to all work together in Visual Studio can be a bit confusing.  I will document the steps I needed to take to get this all working in Visual Studio 2015 SSDT.  Note this will also work with previous version of Visual Studio however I would not go older than Visual Studio 2012.
To make this work you will be unloading and editing the project definition file (it is an xml file).  I strongly suggest you have your project / solution in some sort of source code control system in the event you make a mistake and are unable to open the file.

Start by creating a new Visual Studio database project (I will add mine to GIT later)


Here you can always point to an existing database and import the objects into your current project or start something from scratch.

I will just create a couple of sample objects.
Visual Studio will use a publish profile against a configuration when you publish.  As a standard practice I always create a build configuration that is more meaningful than the included ones.
For example I create a Sandbox build configuration - I use this build configuration when I publish locally.  I also create some kind of final release configuration.  I will create the Sandbox one here for illustration.

Now Right click on the project and click the Publish option.  This will bring up a dialog box where you will enter values specific to that environment.  These are the boilerplate options, any parameters and options you enter later on can be passed in with MSBuild.
Save this profile in the project for ease of use later on, I give them logical names, for example: Sandbox.publish.xml

When Visual Studio publishes a project it will also rely on Pre and Post deployment scripts.  These scripts will executed in the Pre and Post deployment events.  You can do things like populate code tables, drop old objects that you do not want to automatically drop, make data changes, etc.  For today we will just execute a query that will select from the 1 table I created.


So I create a set of folders under a Scripts folder.  These folders will contain a set of special files that will have their Build Action properties set.  Name the folders Post-Deployment and Pre-Deployment.

To get the process to copy and move files depending on the build type specified we need to add a few files to the following folders:
In the Post-Deployment folder I create new script files and call them something like
Custom.Sandbox.PostDeployment.sql
Custom.Release.PostDeployment.sql
(If you need other build configurations you would create additional files as needed to support the other configurations)

What is important is that you can tell which file is for a specific Build Configuration.  If you wanted to do something in Sandbox but not in Release the content of these files would be different.
In the Pre-Deployment folder create files called
Custom.Sandbox.PreDeployment.sql
Custom.Release.PreDeployment.sq

I have added two additional files to help with the demo.  Select_One.sql and Select_Two.sql
Each one of these files will be executed based on the different configuration that is chosen so we can see how our configurations are being applied.
Here is the content of the Select_One.sql file
-- This is from the Select_One script code - it will just select 1.
SELECT 1 FROM dbo.SampleTable


Here is the content of the Select_Two.sql file
-- This is from the Select_Two script code - it will just return A and B.
SELECT 'A' AS rSet
UNION
SELECT 'B' AS rSet

Now we will edit the project file to enable the creation and copying of all the files during the build.
Right click on the project file and choose "Unload Project"  Right click the unloaded project file and select "Edit…"

We are going to add a section to the project's XML file.  This will allow for copying the files over to the output during the build from a configuration.

This is defining the actions to be taken during the build based on the configuration passed in.  When you select Sandbox the build will take our Custom.Release.PreDeployment and Custom.Release.PostDeployment files and copy them over to the generic pre and post deployment files.  So if you want to run some code as part of the pre and/or post deployment the definition of what to run will be in there.

Once you are done save the file and then Right click the project file and Reload it.
Now you can Right click the project and say Build, take note of the project configuration you have selected.
If you were able to successfully build the project you can then go to the Post-deployment folder and add the Script.PostDeployment.sql file by right clicking the folder and saying Add Existing file  -find the Script.PostDeployment.sql and add it, do the same with the Pre-Deployment folder.
It is important to make sure you check the properties of these files.  The property we are concerned with is the Build Action property.  For these two files we want to make sure they are set to PostDeploy and PreDeploy (obviously each based on the name).  This tells the build process to include these files as part of the output and when they should be run
 
Remember that doing something Pre-Deployment means that when you deploy this will be one of the First things to happen.  So if you need to move data during a release from version to version you could do that here before the objects are dropped in a Post-Deployment script.
Now to make it so you can have custom actions based on a build configuration and based on Pre or Post deployment events
edit the Custom.Sandbox.PostDeployment.sql file and add the following:
":r ..\Select_One.sql"  - without the quotes
Save this file

This is a SQLCMD that is telling the build process to move up one path and load up the contents of the Select_One.sql file.  This will write the contents of this file inside the Script.PostDeployment.sql file.  This is one of the files that SSDT will use to create our final output script or to actually publish to a destination server.  The contents are also part of the .dapac file that can also be later deployed using SQLCommand.exe.

Now click on the Sandbox.publish.xml file and click Generate Script. In the output window you should see the script that was created.  If you scroll down though the script you will see a bit of the magic.  You should see the same code that is in the Select_One.sql file!
"-- This is from the Select_One script code - it will just select 1.
SELECT 1 FROM dbo.SampleTable"

You can also make changes to the Custom.Release.PostDeployment.sql file and add the following
":r ..\Select_Two.sql" - without the quotes
Save the file

Change your configuration to Release and click on publish.  This will create a new publish file and will allow you to publish under the current configuration.  Click on generate script.  Browse the created script file and towards the bottom you should see
"-- This is from the Select_Two script code - it will just return A and B.
SELECT 'A' AS rSet
UNION
SELECT 'B' AS rSet"


Once this script file is executed it will run that code.


Now you have a dynamic way to build your SSDT project file from MSBuild passing in a configuration file and have the code execute exactly what you want when you want it.  This is very useful for loading idempotent code table loads, moving data during deployments, cleaning up old objects, etc.

Monday, May 23, 2016

New GIT Repo and code status Visual Studio 2015

For those of you using Visual Studio 2015 with SSDT and are managing your code with GIT a recent update brought a useful, but not easily noticed feature into the mix.  You can now find a status bar located on the bottom of the window that provides some useful information at a glance.  You can now see the repository name, the current branch name, number of pending updates to your project and the push status and count.  This eliminates the need to switch to the Team Explore tab and navigate the menus to check stats and the like.  Thanks Microsoft!

Here you can see that I am connected to the HomeAuto repository and currently working on my development branch where I have 3 pending file changes.


I am not aware of the exact update where this was introduced but for the record I am running version 14.0.25123.00 Update 2.

Microsoft is doing a great job in integrating Visual Studio with various versions of source control products - particularly GIT - I.M.H.O.

Friday, November 13, 2015

Modifying Default Template For Stored Procedures in Visual Studio 2015 SSDT

I have always found it very useful to create database coding standards and making it very easy for myself and others to apply those standards is critical, one of those standards is applying a default comment header block to SQL Server Stored Procedures.
In previous versions of Visual Studio I found it very handy to create a procedure template for new stored procedure creation and distribute that temple to my team members for easy implementation.  In Visual Studio SSDT (SQL Server Data Tools) when adding a new stored procedure to a database project, Visual Studio derives a default stored procedure from a template file.


Here is a sample of what I like to see each time a procedure is created















The scaffolding code is generated from a template located here
C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\SQLDB\Extensions\SqlServer\Items
NOTE: This path is very specific to Visual Studio 2015 SSDT.

The files Create Procedure.sql and Create Procedure.vstemplate are what do the magic.  Make sure you make a backup of the Create Procedure.sql file for safe keeping.

Edit the Create Procedure.sql file and include any comment text or other common items you find yourself typing each time you create a procedure.  For example BEGIN and END statements, setting NOCOUNT ON; NOCOUNT OFF;, etc.

Because of permission issues save this modified file to a DIFFERENT folder
Now delete the original file from the Visual Studio path
Copy the file modified file back into the directory
Restart Visual Studio
Now each time you create a new stored procedure your boilerplate code will be waiting for you.

Monday, January 20, 2014

15 Grammar Goofs That Make You Look Silly
Like this infographic? Get more content marketing tips from Copyblogger.

Friday, December 20, 2013

Database Deployments with SSDT–Data File Sizing

I recently ran into an issue when migrating a database from DataDude to SSDT in Visual Studio 2012.  No matter how hard I tried I could not get the deployment, actually I should say Publish now, to honor the sizing that I defined.  For example: I defined a data file that I assigned to PRIMARY as the only data file for this database.  I made sure to add the File Size and File Growth properties like a good DB guy should, yet when I generated the create script they were just not there.  Here is what I included in the project

ALTER DATABASE [$(DatabaseName)]
    ADD FILE
    (
        NAME = [MyApplication],
        FILENAME = '$(DefaultDataPath)$(DefaultFilePrefix).mdf', SIZE = 50MB , FILEGROWTH = 100MB
    ) TO FILEGROUP [PRIMARY]

However here is what the publish output looked like:

CREATE DATABASE [$(DatabaseName)]
    ON
    PRIMARY(NAME = [MyApplication], FILENAME = '$(DefaultDataPath)$(DefaultFilePrefix).mdf')

Where are my file settings?  How is SSDT ignoring my commands and why would it be doing this?  As it turns out this is expected behavior and it is controlled by a couple of settings.

Open the publish file within SSDT and click on the Advanced settings button.   Scroll down to find “Script File Size” option which is OFF by default.  This setting controls what SSDT does with the 3 properties exposed when creating a data or log file; Size, Maxsize, and Filegrowth.  Checking this value will cause SSDT to script those properties on publish.

image

Now there are a few reasons why this is set to be off by default and you can see the explanation here.

http://connect.microsoft.com/SQLServer/feedback/details/740323/ssdt-published-db-does-not-respect-size-and-filegrowth-properties-when-adding-db-files

The main reason is that by default you would not normally want a production sized database on your Sandbox/ local development machine, makes perfect sense and this is a great option.

Of course during an automated build process you should pass in this Script File size property set to TRUE so SSDT and your build process will honor your file size settings – better still make the sizes properties so you dynamically set them for each build and target!

Monday, October 21, 2013

Changing save location for draft posts – Windows Live Writer

While stating out using Windows Live Writer -http://www.microsoft.com/en-us/download/details.aspx?id=8621 – a tool that allows you to create drafts of your blog posts without knowing HTML in an easy to use client side application – I immediately found a need to change where my drafts were saved.  The goal was to create drafts and save them to a shared location where I could easily review them later.  The first location that came to mind was my Dropbox folder, unfortunately there are no built in options for changing this.  I tried to do the registry hack found here but was unsuccessful so I continued my search.  I ended up by finding a solution that works by using “Symbolic Links”  http://en.wikipedia.org/wiki/Symbolic_link

Check out the solution here: http://www.cto20.com/post/Tips-Tricks-Customize-the-Location-of-Windows-Live-Writere28099s-e2809cMy-Weblog-Postse2809d-Folder.aspx

Thank you Antonio!

The First Post with Live Writer

Here is the first post using Live Writer – http://www.microsoft.com/en-us/download/details.aspx?id=8621 -a tool for composing blog posts in an easy client side tool.  Why am I trying out Live Writer?  Well for starters I wanted a way to write a post and save it for review later.  I find that I actually enjoy reviewing and editing things, from code snippets, designs, even online blog posts.  Quickly writing content and storing it away for later in my online blog just did not seem easy and I find that actually submitting a post triggered a mental button in my head to fire that said “I am done with this…move on”.  The ability to review what I write is important. For starters it will limit the emotional reaction posts that can get me into trouble and reviewing for correct content is never a bad thing.