Colorectal cancer remains one of the most prevalent cancer and a leading cause of cancer related death in the US. Many currently used chemotherapeutic agents are derived from botanicals. Identifying herbal sources, including those from ginseng family, to develop better anti-cancer therapies remains an essential step in advancing the treatment of the cancer. In this article, potential roles of ginseng herbs, especially American ginseng and notoginseng, in colorectal cancer therapeutics are presented. The major pharmacologically active constituents of ginsengs are ginsenosides, which can be mainly classified as protopanaxadiol and protopanaxatriol groups. Structure-activity relationship between their chemical structures and pharmacological activities are discussed. In addition, various steaming temperature and time treatment of the ginseng herbs can change ginsenoside profiles, and enhance their anti-cancer activities. This heat treatment process may increase the role of ginseng in treating colorectal cancer.