PowerShell: Log Manager/Cleanup (Update 1.7)

In my environment, I need to keep an archive of some log files for a period of time, whereas other logs are not so critical, and some applications generate logs which are already compressed into ZIP files daily. So I have all these different requirements for logs. I don’t want them filling up my server hard drives and I don’t want to disable logging, so I wrote this script that could cater to these requirements.

This script is also available to download from the Microsoft TechNet Script Center and the PowerShell Gallery.

Features and Requirements

This script can be configured to do the following:

  • Delete files and folder trees older than X days.
  • Move files and folder trees to another location.
  • Archive files and folder trees as a ZIP file.

The script has been tested on Windows 10, Windows Server 2016, Windows Server 2012 R2, and Windows Server 2008 R2. It has been used to manage Microsoft Deployment Toolkit logs, IIS log files, firewall logs, Minecraft Server logs, IRC client logs and the logs generated by my own scripts posted about here.

The PowerShell -file Parameter

When running the script via Schedule Tasks or the command prompt be sure to use the -file parameter before specifying the script, so you can use “double quotes” for the options that need them, if you do not use -file, then you should use ‘single quotes’.

A Quick Self-Aware Note

I’m totally aware that my script is designed to manage logs and yet it will also generate a log, if required. I really like to get a notification when a server runs a script, so that’s why all of my scripts have this option. Some of my scripts generate a log file that gets overwritten each time they run, so they shouldn’t build up over time, but for everything else this script should help.

Generating an encrypted password file

If you’ve used a previous version I’ve changed how the script handles configuring a password for the log notification e-mail. Specifically the password must now be in an encrypted text file. The advantage of this is that the password will no longer be in plain text, which is a security risk. The downside is that you will now need to generate a password file. The command to do this is pretty simple, but it must be generated on the computer the script will be running on, and as the user used to run the script.

To generate the password file, run the following command in PowerShell. When running the command you will be prompted for a username and password. The username doesn’t matter and can be anything, but the password must be the password you want to use to authenticate to your SMTP server.

$creds = Get-Credential
$creds.Password | ConvertFrom-SecureString | Set-Content c:\scripts\ps-script-pwd.txt

After running the commands, you should have a text file contained the encrypted password. Enter the path and filename for the -pwd switch to configure authenticated e-mail notification.

Configuration

Here are the command line options available to you, with some examples of how to use them.

Command Line Switch Mandatory Description Example
-path Yes The location of the log files you wish to manage. C:\inetpub\logs\LogFiles\W3SVC*\*

OR

\\mdt01\deploymentshare$\Logs

-days Yes Entering 30 will mean that files older than 30 days will be managed. Any number greater than 0

30

-backup No* The location to copy the files to. *This switch is not mandatory, but must be used if a zip file is to be created. F:\Log-Archive

OR

\\nas\archive

-workdir No Specify the working directory for the ZIP file to be created. Ideally it should be local to the server for best performance. E:\scripts
-l No Location to store the optional log file. The name of the log file is automatically generated. E:\scripts\log
-sendto No The email address to send the log to. me@contoso.com
-from No* The email address that the log should be sent from.

*This switch isn’t mandatory but is required if you wish to email the log file.

LogManager@contoso.com
-smtp No* SMTP server address to use for the email functionality.

*This switch isn’t mandatory but is required if you wish to email the log file.

mail01.contoso.com

OR

smtp.live.com

OR

smtp.office365.com

-user No* The username of the account to use for SMTP authentication.

*This switch isn’t mandatory but may be required depending on the configuration of the SMTP server.

example@contoso.com
-pwd No* The location of the file containing the encrypted password of the account to use for SMTP authentication.

*This switch isn’t mandatory but may be required depending on your SMTP server.

c:\scripts\ps-script-pwd.txt
-usessl No* Add this option if you wish to use SSL with the configured SMTP server.

Tip: If you wish to send email to outlook.com or office365.com you will need this.

*This switch isn’t mandatory but may be required depending on the configuration of the SMTP server.

N/A

Change Log

16/10/2017 1.7

  • Changed SMTP authentication to require an encrypted password file.
  • Added instructions on how to generate an encrypted password file.

09/10/2017 1.6

  • Added necessary information to add the script to the PowerShell Gallery.

20/09/2017 1.5

  • Improved log output to be more legible.

21/06/2017 1.4

  • Improved commenting on the code for documentation purposes.
  • Added authentication and SSL options for e-mail notification.

13/06/2017 1.3

  • First public release.

I hope this saves you future disk space headaches. If you’d like to get in touch with me please leave a comment or tweet me.

The Complete Script


<#PSScriptInfo .VERSION 1.7 .GUID 109eb5a2-1dd4-4def-9b9e-1d7413c8697f .AUTHOR Mike Galvin twitter.com/digressive .COMPANYNAME .COPYRIGHT (C) Mike Galvin. All rights reserved. .TAGS Log Manager Cleanup MDT Microsoft Deployment Toolkit IIS Internet Information Services .LICENSEURI .PROJECTURI https://gal.vin/2017/06/13/powershell-log-manager/ .ICONURI .EXTERNALMODULEDEPENDENCIES .REQUIREDSCRIPTS .EXTERNALSCRIPTDEPENDENCIES .RELEASENOTES #>

<# .SYNOPSIS The script can cleanup and optionally archive old logs files. .DESCRIPTION The script can cleanup and optionally archive old logs files. This script can: Delete files and folder trees older than X days. Move files and folder trees to another location, after X days. Archive files and folder tress as a ZIP file, after X days. Please note: to send a log file using ssl and an SMTP password you must generate an encrypted password file. The password file is unique to both the user and machine. The command is as follows: $creds = Get-Credential $creds.Password | ConvertFrom-SecureString | Set-Content c:\foo\ps-script-pwd.txt .PARAMETER Path The root path that contains the files and or folders that the script should operate on. .PARAMETER Days The number of days from the current date that files created during should be untouched. .PARAMETER Backup The location that the files and folders should be backed up to. If you do not set this, a back up will not be performed. .PARAMETER WorkDir The path of the working directory used for ZIP file creation. This should be local for best performance. If you do not set this, the files will not be zipped. .PARAMETER L The path to output the log file to. The file name will be Log-Manager-Log.log. .PARAMETER SendTo The e-mail address the log should be sent to. .PARAMETER From The from address the log should be sent from. .PARAMETER Smtp The DNS name or IP address of the SMTP server. .PARAMETER User The user account to connect to the SMTP server. .PARAMETER Pwd The password for the user account. .PARAMETER UseSsl Connect to the SMTP server using SSL. .EXAMPLE Log-Manager.ps1 -Path C:\inetpub\logs\LogFiles\W3SVC*\* -Days 30 -Backup \\nas\archive -WorkDir E:\scripts -L E:\scripts\log -SendTo me@contoso.com -From Log-Manager@contoso.com -Smtp exch01.contoso.com -User me@contoso.com -Pwd P@ssw0rd -UseSsl With these settings, the script will archive IIS logs files older than 30 days as a ZIP file in \\nas\archive, using the E:\scripts folder as a working directory. The log file of the scritp will be output to E:\scripts\log and emailed using an SSL connection. #>

## Set up command line switches and what variables they map to
[CmdletBinding()]
Param(
    [parameter(Mandatory=$True)]
    [alias("Path")]
    $Source,
    [parameter(Mandatory=$True)]
    [alias("Days")]
    $Time,
    [alias("Backup")]
    $Dest,
    [alias("WorkDir")]
    $Zip,
    [alias("L")]
    $LogPath,
    [alias("SendTo")]
    $MailTo,
    [alias("From")]
    $MailFrom,
    [alias("Smtp")]
    $SmtpServer,
    [alias("User")]
    $SmtpUser,
    [alias("Pwd")]
    $SmtpPwd,
    [switch]$UseSsl)

## Count the number of files that are old enough to work on in the configured directory
$FileNo = Get-ChildItem $Source –Recurse | Where-Object CreationTime –lt (Get-Date).AddDays(-$Time) | Measure-Object

## If the number of the files to work on is zero, do nothing
If ($FileNo.count -ne 0)
{
    ## If logging is configured, start log
    If ($LogPath)
    {
        $LogFile = "Log-Manager-Log.log"
        $Log = "$LogPath\$LogFile"

        ## If the log file already exists, clear it
        $LogT = Test-Path -Path $Log

        If ($LogT)
        {
            Clear-Content -Path $Log
        }

        Add-Content -Path $Log -Value "****************************************"
        Add-Content -Path $Log -Value "$(Get-Date -Format G) Log started"
        Add-Content -Path $Log -Value " "
    }

    If ($LogPath)
    {
        Add-Content -Path $Log -Value "$(Get-Date -Format G) The following objects are older than: $Time days and will be processed:"
        Get-ChildItem $Source | Select Name,LastWriteTime | Out-File -Append $Log -Encoding ASCII
        Add-Content -Path $Log -Value " "
    }

    ## If the zip option was configured, copy the working files to a temp dir and create a zip file, then remove the temp dir and move the zip file
    If ($Zip)
    {
        Add-Type -AssemblyName "system.io.compression.filesystem"
        New-Item -Path $Zip\temp -ItemType Directory
        Get-ChildItem $Source | Where-Object CreationTime –lt (Get-Date).AddDays(-$Time) | Copy-Item -Destination $Zip\temp -Recurse -Force
        [io.compression.zipfile]::CreateFromDirectory("$Zip\temp", "$Zip\Logs-{0:yyyy-MM-dd-HH-mm}.zip" -f (Get-Date))
        Remove-Item $Zip\temp -Recurse
        Move-Item $Zip\Logs-*.zip $Dest
        Get-ChildItem $Source –Recurse | Where-Object CreationTime –lt (Get-Date).AddDays(-$Time) | Remove-Item -Recurse

        If ($LogPath)
        {
            Add-Content -Path $Log -Value "$(Get-Date -Format G) Zip file created and copied to: $Dest"
        }
    }

    ## If the backup directory was configured, copy the files to the backup dir
    If ($Dest)
    {
        Get-ChildItem $Source | Where-Object CreationTime –lt (Get-Date).AddDays(-$Time) | Copy-Item -Destination $Dest -Recurse -Force

        If ($LogPath)
        {
            Add-Content -Path $Log -Value "$(Get-Date -Format G) Contents of the backup location: $Dest"
            Get-ChildItem $Dest | Select Name,LastWriteTime | Out-File -Append $Log -Encoding ASCII
        }
    }

    ## If no backup options were configured, or after doing the previous operations, remove the old files
    Get-ChildItem $Source –Recurse | Where-Object CreationTime –lt (Get-Date).AddDays(-$Time) | Remove-Item -Recurse

    ## If log was configured stop the log
    If ($LogPath)
    {
        Add-Content -Path $Log -Value " "
        Add-Content -Path $Log -Value "$(Get-Date -Format G) Log finished"
        Add-Content -Path $Log -Value "****************************************"

        ## If email was configured, set the variables for the email subject and body
        If ($SmtpServer)
        {
            $MailSubject = "Log Manager Log"
            $MailBody = Get-Content -Path $Log | Out-String

            ## If an email password was configured, create a variable with the username and password
            If ($SmtpPwd)
            {
                $SmtpPwdEncrypt = Get-Content $SmtpPwd | ConvertTo-SecureString
                $SmtpCreds = New-Object System.Management.Automation.PSCredential -ArgumentList ($SmtpUser, $SmtpPwdEncrypt)

                ## If ssl was configured, send the email with ssl
                If ($UseSsl)
                {
                    Send-MailMessage -To $MailTo -From $MailFrom -Subject $MailSubject -Body $MailBody -SmtpServer $SmtpServer -UseSsl -Credential $SmtpCreds
                }

                ## If ssl wasn't configured, send the email without ssl
                Else
                {
                    Send-MailMessage -To $MailTo -From $MailFrom -Subject $MailSubject -Body $MailBody -SmtpServer $SmtpServer -Credential $SmtpCreds
                }
            }

            ## If an email username and password were not configured, send the email without authentication
            Else
            {
                Send-MailMessage -To $MailTo -From $MailFrom -Subject $MailSubject -Body $MailBody -SmtpServer $SmtpServer
            }
        }
    }
}

## End

If you’d like to get in touch with me, please leave a comment or tweet me.

-Mike

Follow Mike on Twitter: @Digressive

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s