If the Paris Agreement targets are to be met, there may be very few years left for policy makers to start cutting emissions. Here we calculate by what year, at the latest, one has to take action to keep global warming below the 2K target (relative to pre-industrial levels) at the year 2100 with a 67% probability; we call this the point of no return (PNR). Using a novel, stochastic model of CO2 concentration and global mean surface temperature derived from the CMIP5 ensemble simulations, we find that cumulative CO2 emissions from 2015 onwards may not exceed 424GtC and that the PNR is 2035 for the policy scenario where the share of renewable energy rises by 2%year-1. Pushing this increase to 5%year-1 delays the PNR until 2045. For the 1.5K target, the carbon budget is only 198GtC and there is no time left before starting to increase the renewable share by 2%year-1. If the risk tolerance is tightened to 5%, the PNR is brought forward to 2022 for the 2K target and has been passed already for the 1.5K target. Including substantial negative emissions towards the end of the century delays the PNR from 2035 to 2042 for the 2K target and to 2026 for the 1.5K target. We thus show how the PNR is impacted not only by the temperature target and the speed by which emissions are cut but also by risk tolerance, climate uncertainties and the potential for negative emissions. Sensitivity studies show that the PNR is robust with uncertainties of at most a few years.