ChIP-seq guidelines and practices of the ENCODE and modENCODE consortia

Abstract

Chromatin immunoprecipitation (ChIP) followed by high-throughput DNA sequencing (ChIP-seq) has become a valuable and widely used approach for mapping the genomic location of transcription-factor binding and histone modifications in living cells. Despite its widespread use, there are considerable differences in how these experiments are conducted, how the results are scored and evaluated for quality, and how the data and metadata are archived for public use. These practices affect the quality and utility of any global ChIP experiment. Through our experience in performing ChIP-seq experiments, the ENCODE and modENCODE consortia have developed a set of working standards and guidelines for ChIP experiments that are updated routinely. The current guidelines address antibody validation, experimental replication, sequencing depth, data and metadata reporting, and data quality assessment. We discuss how ChIP quality, assessed in these ways, affects different uses of ChIP-seq data. All data sets used in the analysis have been deposited for public viewing and downloading at the ENCODE (http://encodeproject.org/ENCODE/) and modENCODE (http://www.modencode.org/) portals.

Figures

Figure 1.

Overview of ChIP-seq workflow and antibody characterization procedures. (A) Steps for which specific ENCODE guidelines are presented in this document are indicated in red. For other steps, standard ENCODE protocols exist that should be validated and optimized for each new cell line/tissue type or sonicator. (*) A commonly used but optional step. (B) Flowchart for characterization of new antibodies or antibody lots. (C) Flowchart for use of antibody characterization assays.

Figure 2.

Representative results from antibody characterization assays. (A) Immunoblot analyses of antibodies against SIN3B that (left) pass quality control (Santa Cruz sc13145) and (right) fail quality control (Santa Cruz sc996). Lanes contain nuclear extract from GM12878 cells (G) and K562 cells (K). Arrows indicate band of expected size of 133 kDa. Molecular weights (MW) are in kilodaltons. (B) Immunoblot analysis of an antibody against TBLR1 (Abcam ab24550) that passes quality control and can be used for immunoprecipitation. Immunoprecipitations (IPs) were performed from nuclear lysates of K562 cells. Arrow indicates band of expected size (56 kDa) that is detected in the input lysate (lane 1) and is efficiently (cf. lanes 3 and 2) and specifically (absent in lane 4) immunoprecipitated. (*) IgG light and heavy chains. (C) Immunofluorescence analyses of antibodies that pass (left) and fail (right) quality control. (D) Immunoprecipitation/mass spectrometry analysis of an antibody against SP1 (Santa Cruz sc-17824). Whole-cell lysates (WCL) of K562, GM12878, and HepG2 were immunoprecipitated, and a band of expected size (∼106 kDa) was detected on a Western blot with SP1 primary antibody. The immunoprecipitation was repeated in K562 WCL, separated on a gel, stained with Coomassie Blue, and the band previously detected on the Western blot was excised and analyzed by mass spectrometry. Peptides were identified using MASCOT (Matrix Science) with probability-based matching at P < 0.05. Subsequent analysis was performed in Scaffold (Proteome Software, Inc.) at 0.0% protein FDR and 0.0% peptide FDR. SP1 protein was detected (along with common contaminants that are often obtained in control experiments) (data not shown) and is highlighted in bold and light blue. (E) Histogram depicting motif fold-enrichment (blue) for all transcription factors for which ENCODE ChIP-seq data is available (85 factors). Enrichments are relative to all DNase-accessible sites and were corrected for sequence bias using shuffle motifs. Motif searches were conducted with a matching stringency of 4–6. Where multiple data sets are available for a factor, the data set with the highest enrichment was counted. Data sets that meet the ENCODE standard of fourfold enrichment (indicated by blue line) were found for 60% of factors. Motif representation, as a percentage of all analyzed peaks, is shown in red for all factors for which a data set exists that exceeds the enrichment standard. A total of 96% of these data sets meet the ENCODE standard of >10% motif representation (red line). All calculations were carried out on peaks identified by IDR analysis (0.01 cut-off).

Figure 3.

Peak counts depend on sequencing depth. (A) Number of peaks called with Peak-seq (0.01% FDR cut-off) for 11 ENCODE ChIP-seq data sets. (B) Called peak numbers for 11 ChIP-seq data sets as a function of the number of uniquely mapped reads used for peak calling. (Inset) Called peak data for the MAFK data set from HepG2 cells, currently the most deeply sequenced ENCODE ChIP-seq data set (displayed separately due to the significantly larger number of reads relative to the other data sets). Data sets are indicated by cell line and transcription factor (e.g., cell line HepG2, transcription factor MAFK). (C) Fold-enrichment for newly called peaks as a function of sequencing depth. For each incremental addition of 2.5 million uniquely mapped reads, the median fold-enrichment for newly called peaks as compared with an IgG control data set sequenced to identical depth is plotted.

Figure 4.

Criteria for assessing the quality of a ChIP-seq experiment. (A) Library complexity. Individual reads mapping to the plus (red) or minus strand (blue) are represented. (B) Distribution of functional regulatory elements with respect to the strength of the ChIP-seq signal. ChIP-seq was performed against myogenin, a major regulator of muscle differentiation, in differentiated mouse myocytes. While many extensively characterized muscle regulatory elements exhibit strong myogenin binding, a large number of known functional sites are at the low end of the binding strength continuum. (C) Number of called peaks vs. ChIP enrichment. Except in special cases, successful experiments identify thousands to tens of thousands of peaks for most TFs and, depending on the peak finder used, numbers in the hundreds or low thousands indicate a failure. Peaks were called using MACS with default thresholds. (D) Generation of a cross-correlation plot. Reads are shifted in the direction of the strand they map to by an increasing number of base pairs and the Pearson correlation between the per-position read count vectors for each strand is calculated. Read coverage as wigglegram is represented, not to the same scale in the top and bottom panels.) (E) Two cross-correlation peaks are usually observed in a ChIP experiment, one corresponding to the read length (“phantom” peak) and one to the average fragment length of the library. (F) Correlation between the fraction of reads within called regions and the relative cross-correlation coefficient for 1052 human ChIP-seq experiments. (G) The absolute and relative height of the two peaks are useful determinants of the success of a ChIP-seq experiment. A high-quality IP is characterized by a ChIP peak that is much higher than the “phantom” peak, while often very small or no such peak is seen in failed experiments.

Figure 5.

Quality control of ChIP-seq data sets in practice. EGR1 ChIP-seq was performed in K562 cells in two replicates. ChIP enriched regions were identified using MACS. However, the cross-correlation plot profiles (A) indicated that both IPs were suboptimal, with one being unacceptable. In agreement with this judgment, ChIP enrichment (C) and peak number (D) also indicated failure. The ChIP-seq assays were repeated (B), with all quality control metrics improving significantly (B,D), and many additional EGR1 peaks were identified as a result. (E) Representative browser snapshot of the four EGR1 ChIP-seq experiments, showing the much stronger peaks obtained with the second set of replicates. (F) Distribution of EGR1 motifs relative to the bioinformatically defined peak position of EGR1-occupied regions derived from ChIP-seq data in K562 cells. Regions are ranked by their confidence scores as called by SPP.

Figure 6.

The irreproducible discovery rate (IDR) framework for assessing reproducibility of ChIP-seq data sets. (A–C) Reproducibility analysis for a pair of high-quality RAD21 ChIP-seq replicates. (D,E) The same analysis for a pair of low quality SPT20 ChIP-seq replicates. (A,D) Scatter plots of signal scores of peaks that overlap in each pair of replicates. (B,E) Scatter plots of ranks of peaks that overlap in each pair of replicates. Note that low ranks correspond to high signal and vice versa. (C,F) The estimated IDR as a function of different rank thresholds. (A,B,D,E) Black data points represent pairs of peaks that pass an IDR threshold of 1%, whereas the red data points represent pairs of peaks that do not pass the IDR threshold of 1%. The RAD21 replicates show high reproducibility with ∼30,000 peaks passing an IDR threshold of 1%, whereas the SPT20 replicates show poor reproducibility with only six peaks passing the 1% IDR threshold.

Figure 7.

Analysis of ENCODE data sets using the quality control guidelines. (A–C) Thresholds and distribution of quality control metric values in human ENCODE transcription-factor ChIP-seq data sets. (A) NSC, (B) RSC, (C) NRF. (D) IDR pipeline for assessing ChIP-seq quality using replicate data sets. (E,F) Thresholds and distribution of IDR pipeline quality control metrics in human ENCODE transcription factor ChIP-seq data sets. (Dashed lines) Current ENCODE thresholds for the given metric, which are NSC > 1.05 (A); RSC > 0.8 (B); NRF > 0.8, N1/N2 ≥ 2 (where N1 refers to the replicate with higher N) (E); Np/Nt ≥ 2 (F).