Principal Splunk Architect

Job Description:

Job Title: Principal Splunk Architect

Corporate Title: Up to Senior Vice President

Location: London

Company Overview:

At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. We do this by driving Responsible Growth and delivering for our clients, teammates, communities and shareholders every day.

Being a Great Place to Work is core to how we drive Responsible Growth. This includes our commitment to being a diverse and inclusive workplace, attracting and developing exceptional talent, supporting our teammates' physical, emotional, and financial wellness, recognizing and rewarding performance, and how we make an impact in the communities we serve.

At Bank of America, you can build a successful career with opportunities to learn, grow, and make an impact. Join us!

Location Overview:

London

Our London office is based just a stone's throw from the magnificent St. Paul's Cathedral on bustling King Edward Street. Here you'll find modern workspaces and a state-of-the-art auditorium space. In addition, we're proud to host an onsite restaurant that shares our commitment to sustainability by providing delicious seasonal menus which have been created with the planet in mind. Make sure to take time for yourself and head up to our rooftop terrace and take in the spectacular views across London. Finally, your physical wellness is well-catered for with our onsite gym facilities and medical centre.

Role Description

Are you a recognised Splunk expert with a passion for operating and continuously improving large-scale observability platforms in a complex global banking environment?

We're seeking a Principal Splunk Engineer to lead the day-to-day operation, stability, performance, and evolution of our enterprise monitoring and observability platforms. This role focuses on ensuring our Splunk and Cribl environments remain resilient, scalable, and effective in supporting mission-critical banking services.

Working within the Operational Intelligence team, you'll act as a senior technical leader responsible for platform reliability, operational excellence, service improvement, and production support across a highly distributed technology estate.

• Own the operational health, stability, and performance of enterprise Splunk and Cribl platforms.
• Serve as the senior technical escalation point for complex platform, ingestion, search, and performance issues.
• Lead troubleshooting and resolution of production incidents, working closely with support and engineering teams.
• Drive platform optimisation initiatives covering search performance, data onboarding, retention, storage efficiency, and operational resilience.
• Partner with Application Production Support, Engineering, and Infrastructure teams to ensure effective monitoring and operational readiness of critical applications.
• Manage platform lifecycle activities including upgrades, maintenance, capacity planning, patching, and service improvements.
• Define and maintain operational standards, runbooks, support procedures, and platform governance practices.
• Develop automation solutions using Ansible and other tools to improve operational efficiency and reduce manual effort.
• Monitor platform utilisation, service health, and operational KPIs, providing meaningful reporting and recommendations for improvement.
• Support major incident investigations, root cause analysis, and problem management activities.
• Mentor engineers and support teams on Splunk, Cribl, observability best practices, and troubleshooting techniques.
• Drive continuous improvement across monitoring, alerting, event management, and operational processes.

• Splunk Core Certified Consultant (or equivalent experience).
• Extensive experience administering and supporting large-scale Splunk Enterprise environments.
• Deep expertise in:
  • Indexer Clustering
  • Search Head Clustering
  • SmartStore
  • Multi-terabyte daily data ingestion
  • Performance tuning and operational resilience
  • Capacity planning and platform lifecycle management
• Strong experience operating and supporting enterprise-scale Cribl deployments.
• Advanced Linux/Unix administration skills.
• Strong automation experience using Ansible and modern engineering practices.
• Experience with Git and CI/CD pipelines.
• Proven ability to troubleshoot and resolve complex distributed-system issues in production environments.
• Strong experience supporting highly available, mission-critical platforms with demanding service-level objectives.

• Banking or Financial Services experience.
• Experience within Application Production Support, Production Engineering, or Site Reliability Engineering teams.
• Knowledge of observability technologies such as Dynatrace and OpenTelemetry.
• Major Incident Management experience.
• ITIL, Change Management, and Problem Management practices.
• Experience operating platforms across large multi-data-centre and cloud environments.
• Experience developing operational metrics, service health reporting, and platform capacity models.