Job Cohen studied public law at the University of Groningen university. He acquired his PHD at Leiden University in 1981. Between 1971 and 1981 he has worked as a research assistant at the Educational Research Bureau of Leiden University, from 1981 through 1993 he has subsequently worked as a senior scientist, as a professor and as the rector of Maastricht University (then called Limburg University).

From 1993 through 1994 Cohen was the secretary of state for the Dutch department of education and science, after which he returned to Maastricht University for his second rectorate. During this period Cohen was a member of the Dutch senate, the last two years of which a parliamentary group chairman. From 1998 through 2001 Cohen was the secretary of state for the Dutch department of justice. On January 17, 2001 Job Cohen was installed as the mayor of Amsterdam.