The FinlogiK statistics collector tool for FxCop is intended to help development teams to track the progress of an FxCop backlog cleanup effort. It is capable of counting total remaining and TODO rule violations in an FxCop 1.35 project.

How does it work?

The tool is intended to count two types of FxCop violations:
  1. The number of "TODO" violations that are flagged for short-term clean-up as part of an immediate backlog (i.e.: violations of rules that have already been activated as decribed here, and
  2. The total number of violations left to fix before the entire backlog has been addressed for all existing FxCop rules.

This counting is accomplished by running FxCop multiple times and evaluating the results of the report produced by FxCop for each execution. In particular, the following operations are carried out by the tool:
  1. The tool removes all TODO exclusions (if applicable) from the target FxCop project file.
  2. The tool runs FxCopCmd.exe to count the TODO violations. Any active violation found in the resulting FxCop report is assumed to represent a TODO task.
  3. The tool modifies the FxCop project file to activate all rules.
  4. The tool re-runs FxCopCmd.exe to count the total violations. Any active violation found in the resulting FxCop report is counted.

N.B.: Exclusions created via SuppressMessageAttribute instances in source code are never counted as violations by the tool.

Installation

  1. Download and install FinlogiK.FxCop.Statistics.Setup.msi. (Installation should be performed under a local administrator account.)
  2. After installation on at least one machine, run the CreateDatabase.sql SQL script from the install directory against a SQL Server 2005 instance in order to create the statistics database.
  3. After database creation, add any users who should be able to collect and view statistics to the public database role.

Client configuration

  1. Launch the FinlogiK.FxCop.Statistics.Windows.exe application.
  2. Use the database configuration dialog to specify the connection string to the statistics database. (If you cannot use the connection string builder, you should still be able to provide a connection string in the following format (without the quotes): "Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=FxCopStatistics;Data Source=SomeSql2005Instance".)

Adding target applications

Before using the tool, you will need to provide a list of applications/projects for which you wish to collect FxCop statistics. To do so, follow these steps:
  1. Launch the FinlogiK.FxCop.Statistics.Windows.exe application.
  2. Click the "Application Profiles" button in the application toolbar.
  3. Enter a row for each application/project for which you wish to be able to collect statistics:
    1. Use the "Name" column to specify the name you wish to use to identify the application within the statistics tool.
    2. Use the "Active" column to indicate whether one should be able to provide new statistics for the application using the tool. This will usually be set to true for a new application.
    3. Use the "Treat all exclusions as TODOs" column to indicate whether all exclusions found the in FxCop project file should be assumed to be backlog cleanup tasks (as opposed to "real" violation suppressions). This will usually be true iff you are using SuppressMessageAttribute for your real exclusions and the FxCop project for your TODO exclusions.
    4. The "FxCop project path" column is a user- and machine-specific path to the FxCop project path for the application. It is an optional value that is only used if you perform a statistics collection on that particular client machine.
    5. The "FxCopCmd.exe path" column is a user- and machine-specific path to FxCopCmd.exe (usually found in C:\Program Files\Microsoft FxCop 1.35\FxCopCmd.exe). It is an optional value that is only used if you perform a statistics collection on that particular client machine.

Specifying TODO prefixes

Unless all the applications for which you will be using the statistics tool are flagged to treat all exclusions found in the FxCop project file as TODOs, you will need to help the tool identify which of the in-project exclusions are TODOs as opposed to real exclusions. This is done by specifying a list of prefixes that the tool will attempt to match in the last note recorded against any given in-project exclusion.
To provide a prefix list, follow these steps:
  1. Launch the FinlogiK.FxCop.Statistics.Windows.exe application.
  2. Click the "TODO prefixes" button in the application toolbar.
  3. Enter a row for each prefix (case-insensitive) that should identify a TODO exclusion.

For example, the current TODO prefix list in use at FinlogiK is the following:
  1. ?
  2. pas le temps
  3. temporary
  4. to check
  5. to do
  6. todo
  7. too many problems
  8. too many to fix
  9. will be automatically converted
  10. will be fixed

Since the appropriate prefix list may evolve as developers add new exclusions to your FxCop projects, you should consider periodically reviewing your in-project exclusions in order to identify any new TODO prefixes that may have crept in.

Running an analysis using the tool UI

Unless you are already using automated builds for your target projects, you will probably end up running statistics collection analyses from the tool UI. To launch an analysis run from the UI, follow these steps:
  1. Launch the FinlogiK.FxCop.Statistics.Windows.exe application.
  2. Click the "Run Analysis..." button in the application toolbar.
  3. Select the application/project for which you wish to perform the analysis.
  4. Click the "Start" button to initiate the analysis run.
  5. Once the analysis run has completed, a message box will be shown indicating whether the analysis was successful and inviting you to view the resulting statistics should you wish to do so immediately.

Running an analysis from an automated build

The tool may be launched in command line mode by providing the following arguments (in this order):
  1. The project name (required),
  2. The statistics database connection string (optional),
  3. The full path to FxCopCmd.exe (optional), and
  4. The full path to the FxCop project file to be used in the analysis (optional).

The command line arguments are read by position, so you should provide values for all optional arguments if you provide values for any of them. If you do not provide the optional values, they will be read from the tool configuration for the build machine and user.

In practice, you probably won't want to perform a statistics collection at every automated build, particularly if you're using continuous integration. At FinlogiK, we've set things up so that we only run FxCop stats collection once per day for any given project.

Viewing analysis results

To view analysis results, open the tool UI and click the "View statistics" button in the toolbar. Select the application and statistics run for which you wish to view statistics. (The analysis runs are sorted in descending chronological order, and the timestamp shown is UTC.)

The four tabs represent the summary and detailed results for each of the "total" and "TODO" violation categories. The summary results show violation counts by rule category, and the detailed results show violation counts by rule.

If you wish to modify the category mappings (for example, to merge multiple categories of custom rules into one category) and/or sorting for the summary results, this can be done by manually editing the CategoryGroups and RuleCategories tables in the statistics database. The current version of the statistics tool does not include a UI for editing this data.

Known issues

This tool was developed for internal use at FinlogiK, and its functionality set is pretty close to the absolute minimum that we need for tracking our FxCop backlog cleanup efforts. There's quite a bit we would have liked to change before distributing a version here on CodePlex, but that would probably have delayed the first public release more of less indefinitely. So... Instead of no release at all, what you see here is exactly the same beastie we're using internally, warts and all.

We've added some of those those warts to the Issue Tracker for this project. Unfortunately, given limited resources, it's unlikely that we'll be addressing these issues any time soon, but at least you'll know that they bug us too... ;) If there's some change that you absolutely can't live without, please feel free to download the source code and modify it as necessary for your internal use.

Last edited Nov 10, 2009 at 6:45 PM by dgouin, version 4