The company was started by Robert Haber in 1978 as the College Media Journal, a bi-weekly trade magazine aimed at college radio programmers in Great Neck, NY. The first issue was published on March 1, 1979, and featured Elvis Costello on the cover. Staff would often describe these early issues as "a bunch of photocopies stapled together."

A year and a half later, the magazine was able to create the first annual CMJ Music Marathon, which was named after the New York City Marathon, which was held at around the same time of year. About 100 people attended, and there were no showcases.

In 1982, the magazine officially rebranded as CMJ New Music Report.

CMJ continued to grow, and by the 90s, Haber and the CMJ staff would often find themselves wanted to stay connected to the college radio scene, but felt unable or unwilling to pay the high price for a subscription to a trade publication. To fill the need, CMJ created CMJ New Music Monthly in 1993. This consumer publication featured interviews, reviews, and special features. It was also the first magazine to regularly include a CD of music. It was available on newsstands and via subscription.

The company got caught up in the internet bubble in the late 90s when Rare Medium Group purchased CMJ in November 1999. Rare owned interest in iFace, ePrize, LiveUniverse, and the ChangeMusic Network (the latter of which, CMJ would become a subsidiary). Rare would also move CMJ out of Great Neck and into a new office in New York, NY. The internet conglomerate fizzled by 2001, and Haber purchased a debt-saddled CMJ back from Rare.

The shift back to being an independent company proved to be difficult. New Music Monthly saw several issues be "subscriber only" and unavailable on newsstands. That year's Music Marathon in New York City was intended to be a comeback for the now independent media company with hundred of bands, many panelists, and thousands of attendees scheduled to attend between September 13–16 with an ill-picked tagline in hindsight of "A Killer Event." The tagline proved to be tragically ominous as the events of September 11, 2001, would end up canceling the event. A scaled back version of the Marathon ended up taking place a month later.

Minor scandals soon followed. CMJ was accused of manipulating their charts in order to push their own compilation into the Top 200; however, CMJ claimed it was an accident and the compilation was only used as a placeholder.[3] This resulted in CMJ changing the name of their New Music Report compilation from Certain Damage to On Air. In addition, the magazine was criticized at the time by many in the independent music community for focusing too much on major label acts, which resulted in Beggars Group pulling ads from the publication.

By 2004, New Music Monthly was off newsstand completely and issues were often sent out sporadically, which made it hard to attract new subscribers. New Music Report went from being a weekly publication to only being sent out every other week (the published charts were available only online for the off-weeks).

Though the final issues were sent out in 2008, New Music Monthly officially stopped publishing on June 20, 2009. New Music Report would soon follow.

While the magazines were faltering, CMJ entered into an agreement with Metropolitan Talent for a proposed merger. This resulted in Metropolitan giving CMJ $600,000 in "stay alive money," but ended in a lawsuit when the deal fell apart and CMJ was instead sold to Adam Klein (former CEO of eMusic) and Abaculi Media instead in 2014. In a move described as "devastating," founder Bobby Haber was let go by Abaculi shortly thereafter.

By 2016, CMJ was no longer putting on the annual Music Marathon, and staff stopped getting paid in October 2015, which eventually lead to a lawsuit and Klein's bankruptcy.

In 2019, Amazing Radio purchased the CMJ brand with the hope of reviving the Music Marathon. Offerings of various artists were placed online with at home performances due to the worldwide COVID-19 pandemic.

CMJ New Music Report

CMJ New Music Report published top-30 lists sent to them by radio stations, which subscribed at a cost of a few hundred dollars a year. The magazine moved to an online only format and was released weekly as a digital PDF magazine until it folded in 2017.

On January 5, 2004, CMJ New Music Report published a 25th anniversary double issue led by an editorial staff that included editor-in-chief Kevin Kerry Boyce, and managing editors Louis Miller and Doug Levy. The issue featured The White Stripes on the cover in a photograph captured by art director Drew Goren; the magazine named the band's 2003 release, Elephant, its Album of the Year.

Many musicians from the New York City indie rock community have worked on staff at CMJ over the years, including members of acts such as Parts and Labor, Poingly, and The Airborne Toxic Event.

CMJ Music Marathon

From 1980 through 2015, staff organized the CMJ Music Marathon, a convention and music festival, each autumn, in New York. A second festival, the CMJ Rock Hall Music Fest, took place in Cleveland in 2005 and 2006; in April 2007, organizers canceled the event, citing strains on financial and staffing resources.

CMJ New Music Monthly

CMJ New Music Monthly was a monthly music magazine with interviews, reviews, and special features published from 1993 to 2009. Each issue included a compact disc with 15 to 24 songs by well established bands, unsigned bands, and everything in between. As of issue 156 (1112 using the CMJ New Music Report numbering), dated June 20, 2009, the magazine ceased operation, and subscribers had their remaining issues replaced by the CMJ 

New Music Report with a music compilation available online. By April 2010, it stopped delivering CMJ New Music Report to its subscribers.