Drop folder cleanup. How to achieve it in few easy steps?

drop5

If you use Team Foundation Server or Visual Studio Online automatic build, you probably know that the output of your build may contain large number of files that are not necessarily required. A build drop location in many cases may contain hundreds or even thousands of files. Because of that it may be extremely difficult to find the files you really need in the build drop. It can also lead to other problems, like lack of storage space to accommodate new builds.

One of the solutions to that problem is a custom build activity that will clean the drop folder from all the unnecessary files:

Using Visual Studio 2013 create an empty Class Library project and rename default Class1.cs to something more meaningful like DropCleanupActivity.cs.

Make sure your project references the following assemblies:

  • Activities
  • TeamFoundation.Build.Client (C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.TeamFoundation.Build.Client\v4.0_12.0.0.0__b03f5f7f11d50a3a)
  • TeamFoundation.Build.Workflow (C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\PrivateAssemblies)

Define two arguments in the class:

#region Arguments
  [RequiredArgument, DisplayName("Binaries Directory"), Description("Path to binaries directory.")]
public InArgument<string> BinariesDirectory { get; set; }
  [RequiredArgument, DisplayName("Output Required File Names"), Description("StringList with file names.")]
public InArgument<StringList> OutputRequiredFileNames { get; set; }

#endregion Arguments

BinariesDirectory will point to the folder where all the output files are stored. In other words this will be the location that needs to be cleaned. OutputRequiredFileNames will hold the list of file names that should not be deleted – those are the files we are expecting to be in the drop location.

Next override Execute method:

protected override void Execute(CodeActivityContext context)

{

}

Now let’s add some logic to our activities. First, add a method that will remove empty directories recursively:

private void RemoveDirectory(string directory, CodeActivityContext context)
{
    string[] subDirecotries = Directory.GetDirectories(directory, "*", SearchOption.TopDirectoryOnly);

    if (subDirecotries != null && subDirecotries.Length > 0)
    {
        foreach (string subDirectory in subDirecotries)
        {
            RemoveDirectory(subDirectory, context);
        }
    }
    else
    {
        Directory.Delete(directory);
        context.TrackBuildMessage(string.Format(@"Directory removed: '{0}'.", directory), BuildMessageImportance.High);
    }
}

Now we need a logic that will clean the drop location from the unnecessary files. We will specify file names that we need in the build definition in the process section. The activity will remove all the files and empty directories leaving only the required files.

While the activity is iterating though files and folders it will log information to the build by the TrackBuildMessage method. This way it will be easy to check which files are required and which were removed during the build process.

Below is the Execute method body:

// Obtain the runtime value of arguments
string binariesDirectory = context.GetValue(this.BinariesDirectory);
StringList outputRequiredFileNames = context.GetValue(this.OutputRequiredFileNames);

string currentItem = string.Empty;
try
{
    // Set default values for arguments
    if (string.IsNullOrEmpty(binariesDirectory)) binariesDirectory = string.Empty;
    if (outputRequiredFileNames == null) outputRequiredFileNames = new StringList();

    if (!string.IsNullOrEmpty(binariesDirectory))
    {
        // Required directories will be kept here
        List<string> requiredDirectories = new List<string>();

        // Temporary variable for file
        FileInfo fileInfo = null;

        context.TrackBuildMessage(string.Format("Cleaning binaries directory '{0}'.", binariesDirectory), BuildMessageImportance.High);

        // Log reguired file names
        foreach (string requiredFile in outputRequiredFileNames)
        {
            context.TrackBuildMessage(string.Format("Required output file name: '{0}'.", requiredFile), BuildMessageImportance.High);
        }

        // Get all files
        string[] allFiles = Directory.GetFiles(binariesDirectory, "*", SearchOption.AllDirectories);

        // get all directories
        string[] allDirectories = Directory.GetDirectories(binariesDirectory);

        // Iterate through all file and remove not required files
        context.TrackBuildMessage("Deleting not required files.", BuildMessageImportance.High);
        foreach (string file in allFiles)
        {
            currentItem = file;
            fileInfo = new FileInfo(file);

            if (!outputRequiredFileNames.Contains(fileInfo.Name))
            {
                // File is not required - delete it
                fileInfo.Delete();
                context.TrackBuildMessage(string.Format("File removed: '{0}'.", file), BuildMessageImportance.High);
            }
            else
            {
                // File is required. Remeber the directory
                if (!requiredDirectories.Contains(fileInfo.Directory.FullName) && !fileInfo.Directory.FullName.Equals(binariesDirectory))
                {
                    requiredDirectories.Add(fileInfo.Directory.FullName);
                }
            }
        }

        // Log required direcories
        context.TrackBuildMessage("Calculating required directories.", BuildMessageImportance.High);
        foreach (string directory in requiredDirectories)
        {
            context.TrackBuildMessage(string.Format(@"Required directory: '{0}'.", directory), BuildMessageImportance.High);
        }

        // Iterate through all direcories
        context.TrackBuildMessage("Deleting not required directories.", BuildMessageImportance.High);
        foreach (string directory in allDirectories)
        {
            currentItem = directory;
            if (!requiredDirectories.Contains(directory))
            {
                // Remove subdirecotries in a directory and directory
                RemoveDirectory(directory, context);
                context.TrackBuildMessage(string.Format(@"Directory removed: '{0}'.", directory), BuildMessageImportance.High);
            }
         }
     }
     else
     {
          context.TrackBuildMessage("Server path to installer configuration file not set. Customization name will not be set.", BuildMessageImportance.High);
     }
}
catch (Exception exc)
{   context.TrackBuildError(string.Format(@"DropFolderCleenupActivity failed on item '{0}'. Exception message: {1}.", currentItem, exc.Message));
throw exc;
}

At this point the project should build successfully. Make sure the assembly has the strong name – this will be needed in order to add the assembly to GAC. Later adding the assembly to GAC will be required.

What is left is to modify the build process template to include our custom cleanup activity.

Create a new XAML Build definition and make sure that TfvcTemplate.12.xaml is selected in the Process section. Click Download to download the process template. You can customize any arbitrary process template – in this case I’m using a standard TfvcTemplate.12.xaml

Open TfvcTemplate.12.xaml in Visual Studio 2013 and select Arguments from the bottom toolbar. Create a new argument called DropFolderCleanupRequiredFiles and make sure Direction equals In. Argument type should be StringList.

Once this is added find Metadata argument. Click the ellipsis button on the right:

drop1

In the Process Parameter Metadata Editor click Add and fill the form in the following way:

drop2

Make sure that Parameter Name is equal to the previously created Argument Name. (DropFolderCleanupRequiredFiles). You can enter any Display Name you want, but it is a good idea to add a number at the beginning, as parameters in the process form are sorted by it. You can also set whatever Category you want but the prefix (here #900) tells Visual Studio the order, in which categories should be displayed. In the Editor field enter Microsoft.TeamFoundation.Build.Controls.WpfStringListEditor – this editor deals with required file names easily and intuitively.

Now we are only missing our build workflow activity. First, install the assembly with drop folder cleanup activity in GAC. You can do this from elevated command prompt with the following command:

gacutil /if c:\Tfs.SampleActivities\Bin\Debug\Tfs.SampleActivities.dll

Note: Make sure you use gacutil for the .Net 4 or above. Adjust the paths to your needs.

In the Build process workflow go to Overall build process -> Run on agent -> Try -> Finally -> Perform Final Actions On Agent. There should be two activities: Copy binaries to drop and Reset the Environment:

drop3

We need to add our drop folder cleanup activity just before Copy binaries to drop. In order to do so, right click on the toolbox in the General section and choose Choose Items…:

drop4

In the Choose Toolbox Items dialog click Browse… and select the assembly from GAC (C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Tfs.SampleActivities\v4.0_1.0.0.0__2fb10a79391e423b\Tfs.SampleActivities.dll). Valid activities from the chosen assembly and should be selected automatically. Click Ok to add activities to the toolbox:

drop5

Drag the activity in the toolbox just before Copy binaries to drop. Add the additional activity just before the newly created DropCleanupActivity – GetEnvironmentVariable. When asked, set the type as string:

drop6

In order to pass the binaries location to the DropCleanupActivity we will need to extract it from the build. For that we will use GetEnvironmentVariable<string> activity with a special configuration.

First, we will create a temporary variable. From the bottom toolbar choose Variables and create a new variable called BinariesDirectory. Set the type to string.

Right click GetEnvironmentVariable<string>, choose Properties and configure it as follows:

  • Set value of a property called Name to TeamFoundation.Build.Activities.Extensions.WellKnownEnvironmentVariables.BinariesDirectory
  • Set value of property called Result to BinariesDirectory (temporary variable)

Binaries location is now available, so we can configure the cleanup activity.

Right click the cleanup activity and choose Properties.

  • for the Binaries Directory set BinariesDirectory (temporary variable)
  • for the Output Required Files set DropFolderCleanupRequiredFiles (argument we created few moments ago)

Save the TfvcTemplate.12.xaml file. It is now ready to be deployed to TFS, but it must be added to the version control so that it could be used. I placed mine file in the custom Build Templates folder directly in the project source control ($/TfsTest_VisualStudio2013/Build Templates/TfvcTemplate.12.xaml).

Now in the Build definition in Process section choose New… and find the customized build process template:

drop7

Configure the build definition and list all required files in the Drop Folder Cleanup section:

drop88

Lastly, make sure that assembly containing custom activity is on TFS build machine. It can be installed in GAC, added to the known reference folder or added to the version control.

The simplest option is to place the assembly in the version control and set the build controller properties to look for the custom build assemblies in a certain location – this will also work with Visual Studio Online Hosted Build Controller:

drop8

When you run the build you should see the logs and your drop folder location will not contain any unnecessary files.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s