To test your plugins, you can use unit tests. There are some examples in the project DevTestMethods.

To test multiple samples, you can use the -test command line argument when starting CrypTool. A new ribbon dock will be shown with "Demo Controls". When you "Play Demo", you have to select a logfile where to save the test results and to select the samples which shall be tested. CT2 will automatically run these samples for a couple of seconds and will save all error and warning messages to the logfile.