We use nuclear magnetic resonance (NMR), high-resolution x-ray, and neutron scattering studies to study structural and magnetic phase transitions in phosphorus-doped BaFe2(As1-xPx)2. Previous transport, NMR, specific heat, and magnetic penetration depth measurements have provided compelling evidence for the presence of a quantum critical point (QCP) near optimal superconductivity at x=0.3. However, we show that the tetragonal-to-orthorhombic structural (Ts) and paramagnetic to antiferromagnetic (AF, TN) transitions in BaFe2(As1-xPx)2 are always coupled and approach TN ≈ Ts ≥ Tc (≈29 K) for x=0.29 before vanishing abruptly for x≥0.3. These results suggest that AF order in BaFe2(As1-xPx)2 disappears in a weakly first-order fashion near optimal superconductivity, much like the electron-doped iron pnictides with an avoided QCP.