In this paper, we present DUBLIN, a pixel based model for visual document understand ing that does not rely on OCR.DUBLIN is pretrained on a large corpus of document images with novel tasks that enhance its visual and linguistic abil ities.Our model demonstrates the potential of OCR-free document processing and opens new avenues for applications and research.