I finally managed to get back to the office (damn pandemic) and monitor voltage. The base64 error happens when input voltage drops to 3.2V (and might be lower without my multimeter noticing), so I think it is pretty likely that is the issue. For whatever reason this probably triggers my other I2C error as well.