The detection of a primordial B-mode signal in the polarization of the cosmic microwave background (CMB) would be the smoking gun for inflation. As such it would constitute the single most important discovery in physics in the last several decades, probing still- uncharted physics territory, including the first instants of the Universe and quantum gravity. While recent results by the BICEP2 experiment rallied enthusiasm in the community by suggesting that this detection may have already been achieved, the analysis of all-sky data from ESA's Planck satellite mission has shown that this breakthrough is still out of reach, because of the most severe, ever-present polarization foreground: Galactic dust. It is now clear that our path to the inflationary B-modes passes through successfully modeling and subtracting the polarized emission of interstellar dust at high accuracy. A critical piece of this puzzle is the 3-d structure of the magnetic field threading dust clouds, which cannot be accessed through microwave observations alone, since they record integrated emission along the line of sight. Instead, observations of a large number of stars at known distances in optical polarization, tracing the same CMB- obscuring dust, can map the magnetic field between them. The Polar Area Stellar Imaging in Polarization High Accuracy Experiment (PASIPHAE) will deliver such a map combining novel-technology wide-field-optimized optical polarimeters and an extraordinary commitment of observing time by the Skinakas observatory in Crete and the South African Astronomical Observatory. PASIPHAE will measure, at unprecedentedly high accuracy, the polarimetric properties of several millions of stars n the areas of the sky targeted by CMB experiments, achieving a 1000-fold improvement over the current state of the art. Such a map would not only boost CMB polarization foreground removal, but it would also have a profound impact in a wide range of astrophysical research, including interstellar medium physics, high-energy astrophysics, and galactic evolution.