Circulating tumor cells (CTCs) in the blood of cancer patients are of high clinical relevance. Since detection and isolation of CTCs often rely on cell dimensions, knowledge of their size is key. We analyzed the median CTC size in a large cohort of breast (BC), prostate (PC), colorectal (CRC), and bladder (BLC) cancer patients. Images of patient‐derived CTCs acquired on cartridges of the FDA‐cleared CellSearch® method were retrospectively collected and automatically re‐analyzed using the accept software package. The median CTC diameter (μm) was computed per tumor type. The size differences between the different tumor types and references (tumor cell lines and leukocytes) were nonparametrically tested. A total of 1962 CellSearch® cartridges containing 71 612 CTCs were included. In BC, the median computed diameter (CD) of patient‐derived CTCs was 12.4 μm vs 18.4 μm for cultured cell line cells. For PC, CDs were 10.3 μm for CTCs vs 20.7 μm for cultured cell line cells. CDs for CTCs of CRC and BLC were 7.5 μm and 8.6 μm, respectively. Finally, leukocytes were 9.4 μm. CTC size differed statistically significantly between the four tumor types and between CTCs and the reference data. CTC size differences between tumor types are striking and CTCs are smaller than cell line tumor cells, whose size is often used as reference when developing CTC analysis methods. Based on our data, we suggest that the size of CTCs matters and should be kept in mind when designing and optimizing size‐based isolation methods.