A wide range of approaches (>200) have been applied towards in silico prediction of transcription factor binding sites (TFBS). Tools differ in algorithmic approach, organism, the ability to integrate cross-species information, the ability to integrate sources from different instruments, the statistical methods employed, and the parameters utilized in the search. The large variety makes it difficult to determine which methods perform the best for the system being studied. To address these problems, we developed the motif tool assessment pipeline (MTAP). MTAP is an integrated framework for the execution and evaluation of TFBS detection algorithms. MTAP integrates multiple tools by converting arbitrary input, output and workflows into a common data model. This allows tools to be integrated and evaluated from the perspective of biological system reconstruction. This common model enables the development of statistical approaches to compare TFBS detection algorithms, and computational infrastructure for handling large datasets.