A growing number of tools and procedures claim to properly de-identify image data. Based on our decade of experience managing the Cancer Imaging Archive (TCIA) on behalf of NCI, we developed a DICOM dataset that can be used to evaluate the performance of de-identification algorithms. DICOM objects were selected from datasets published in TCIA.  Synthetic Protected Health Information (PHI) was generated and inserted into selected DICOM Attributes to mimic typical clinical imaging exams.  The DICOM Standard and TCIA curation audit logs guided the insertion of synthetic PHI into standard and non-standard DICOM data elements. An answer key was created based on our knowledge of the placement of synthetic data and the DICOM standard’s guidelines for what actions should be taken in regard to the synthetic PHI.  A TCIA curation team tested the utility of the evaluation dataset and answer key.