Continuous Integration: Move Unicorn Files Outside of Website

In the previous blog post about Unicorn and Continuous Integration, Unicorn Sync: YML File Used by Another Process, I have mentioned some issues about the Unicorn Sync step and a temporary solution that helped us a lot. However, we had to improve the process because, even though we are talking about a DEV server, it is still better not to kill the website in IIS during the deploy.

The solution is to change the Deployment step in Octopus by replacing the Pre-deployment PowerShell script with a Deployment PowerShell script which moves the Unicorn files to a different folder, outside of the website. By doing this during the deploy, the YML files do not get to be used by the w3wp process, so it is possible to move them to a different location. Before we move the files, we ensure that the folder where we move them is empty, thus avoiding old files still being used. Below is the script we use:

$folder = "<Path>\Website\Unicorn"
$newfolder = "<Path>\Data\Unicorn"

if (Test-Path $newfolder) {
    Get-ChildItem -Path $newfolder -Force -Recurse |
    Sort-Object -Property FullName -Descending |
    Remove-Item -Recurse -Force
}

if (Test-Path $folder) {
    Get-ChildItem -Path $folder -Recurse | 
    Move-Item -Destination $newfolder -Force
}

Then, the only thing we need to change is the Unicorn.config file so that the targetDataStore points to the new location.

Unicorn Sync: YML File Used by Another Process

I am currently working on a Sitecore 8.2 solution, for which we have set up Continuous Integration on a DEV server. This is achieved by using Team City and Octopus, and the deploy is performed upon push to GIT ’develop’ branch. In order to avoid transferring Sitecore items manually via packages, we have set up Unicorn and we have a step in the deploy process that performs the Unicorn Sync. The YML files are stored in a folder outside of the VS solution, but we have a .nuspec file that specifies that this folder should be part of the package for deploy. Then, on the DEV server, the deploy package is extracted under our Website, and this includes also the Unicorn folder.

We chose not to clean the whole Website directory before performing the deploy, but we do need the Unicorn folder to be clean, otherwise obsolete files will not be deleted. In order to do this, we have a Pre-deployment PowerShell script that deletes all the YML files:

$folder = "<Website-Path>\Unicorn"
if (Test-Path $folder) {
    get-childitem -path $folder -recurse | 
       remove-item -force -recurse
}

But sometimes, this script fails to run, causing the entire deploy to fail, and the error is this:

Remove-Item : Cannot remove item <Website-Path>\Unicorn\<Path-To-File>\<Filename>.yml: The process cannot access the file ‘<Filename>.yml’ because it is being used by another process.

From the Sitecore log, we can see that the process using the file is w3wp. This does not happen all the time, and never with the same file. The manual fix for this is to stop the corresponding Application Pool and Site from IIS , delete the file, then start again the Application Pool and the Site. The next run of the deploy process will work, but this is not a fix to the issue. The probable cause for the blocked file is the fact that it is stored under the Website, but configuring the Unicorn to store it outside of the Website folder needs a bit more investigating. The quicker solution that I found was to modify the PowerShell script, so that it stops and starts the Application Pool and the Site that we specify:

$folder = "<Website-Path>\Unicorn"
$appPoolName = "<AppPoolName>"
$websiteName = "<WebsiteName>"

Import-Module WebAdministration
Stop-WebSite $websiteName

if((Get-WebAppPoolState $appPoolName).Value -ne 'Stopped')
{
  Stop-WebAppPool -Name $appPoolName
}

if (Test-Path $folder) {
    get-childitem -path $folder -recurse | 
        remove-item -force -recurse
}

if((Get-WebAppPoolState $appPoolName).Value -ne 'Started')
{
  Start-WebAppPool -Name $appPoolName
}

Start-WebSite $websiteName

No deploys have failed since!