Data Architect

The Data Architect is responsible for designing, building, and evolving the enterprise data foundation supporting the Firm’s artificial intelligence, analytics, and business applications.  This position translates complex enterprise data requirements into practical, scalable architectures and delivers reliable, governed data that technology teams and business stakeholders can use with confidence.  This is a technical role with primary responsibility for enterprise data modeling, lakehouse architecture, data integration, governance, and data quality.  The Data Architect will design and implement solutions within Azure Databricks, develop data pipelines and models, and establish standards and reusable patterns for the Firm’s growing data environment. The position will partner closely with AI Engineering, DevOps, Information Security, Knowledge Management, and other technology and business stakeholders to ensure data is accessible, secure, traceable, and appropriately governed.

KEY RESPONSIBILITIES

• Lead the design, implementation, and ongoing evolution of the Firm’s Azure Databricks data architecture, including lakehouse layers, storage, data models, integration patterns, and the roadmap from current-state systems to the target architecture.
• Partner with attorneys, practice groups, business teams, and technology stakeholders to identify priority data requirements, define data products, and establish measurable standards for data quality, freshness, availability, and usability.
• Design conceptual, logical, and physical data models for enterprise information, including client, matter, people, document, financial, and operational data, establishing consistent definitions, identifiers, relationships, and standards in partnership with data owners.
• Design, build, and maintain scalable data ingestion and transformation pipelines using Python, SQL, Apache Spark, Delta Lake, and related technologies, selecting appropriate batch, incremental, change-data-capture, or streaming approaches based on business requirements.
• Integrate data from enterprise databases, APIs, files, document repositories, and other systems through supported interfaces, including the development of source-to-target mappings, data contracts, reconciliation processes, and controls for schema changes and deletions.
• Design and administer data governance within Unity Catalog, including catalogs, schemas, ownership structures, access policies, lineage, classification, retention, and audit requirements.
• Partner with Information Security and data owners to design, implement, and validate access controls that appropriately reflect source-system permissions, client and matter restrictions, ethical walls, and other confidentiality requirements.
• Establish and maintain data quality standards, automated validation, monitoring, recovery procedures, and service expectations. Troubleshoot data and pipeline failures and optimize reliability, query performance, compute utilization, storage, and overall platform costs.
• Develop curated datasets and governed data interfaces supporting analytics, AI agents, retrieval-augmented generation, and other AI-enabled workflows while maintaining appropriate source traceability and access controls.
• Design and implement Lakebase PostgreSQL data stores supporting agentic applications, including persistent agent state, checkpoints, and memory, with appropriate user and matter isolation, transactional access patterns, retention, and recovery.
• Establish reusable architectural standards, technical documentation, and engineering patterns and provide technical guidance, design review, code review, and mentorship to AI Engineers and other technical team members.
• Partner with DevOps and other technology teams to support secure environments, automated deployments, development/test/production processes, monitoring, operational readiness, and long-term platform supportability.
• Remain current on developments in data architecture, Azure Databricks, cloud data engineering, AI data infrastructure, governance, and related technologies, recommending enhancements where appropriate.

PREFERRED SKILLS & KNOWLEDGE

• Master’s degree in Computer Science, Data Engineering, Information Systems, or a related discipline.

•    Experience with master and reference data management, entity resolution, data stewardship, and enterprise data governance across complex systems.

• Experience developing governed data foundations for artificial intelligence, machine learning, AI agents, and retrieval-augmented generation (RAG), including document preparation, metadata management, vector search, or knowledge graphs.

•   Experience with Databricks SQL, business intelligence integrations, semantic models, and governed datasets supporting enterprise reporting and analytics.  Experience with infrastructure as code and automated deployment technologies such as Terraform, Azure DevOps, or GitHub Actions.

• Experience modernizing legacy data platforms or migrating enterprise data environments to cloud-based architectures.

•   Experience working in legal services, professional services, financial services, or another environment involving highly sensitive information, complex confidentiality requirements, and sophisticated access controls.

•   Experience in technical consulting or another business-facing technology delivery role requiring direct engagement with business stakeholders and senior leaders.

• Ability to sit and stand for extended periods.

• Ability to lift up to 15 pounds.

Pillsbury Winthrop Shaw Pittman LLP is an Equal Opportunity Employer.

If you require an accommodation in order to apply for a position, please contact us at PillsburyWorkday@pillsburylaw.com.