Staff Software Engineer with over 15 years of experience in designing and building scalable and reliable systems. Proficient in a wide range of technologies including Java, Python, and AWS. Passionate about mentorship and leveraging AI to solve complex problems. When not at the keyboard, I am a proud father of three, a board game enthusiast, and an avid self-hoster.
Staff engineer on a model-serving platform team, building and operating the self-hosted small language model inference stack and leading cost, reliability, and migration initiatives across partner teams.
Designed and delivered the company’s first self-hosted SLM inference platform end-to-end, from architecture review to production: vLLM based, drop-in OpenAI Chat Completions API, LoRA adapter serving, FP8 quantization, schema-constrained structured outputs, per-use-case isolation, and KEDA autoscaling.
Migrated production workloads off external LLM vendors onto the platform, saving over $7M annually, including an 87% reduction (~$5.8M/year) on a 60M request/day generation workload and a 93% reduction (~$1.34M/year) on a fine-tuned ranking model ramped to 100% of traffic.
Delivered inline query embeddings on the live job-search path at ~20ms p95, a latency budget third-party embedding APIs (~500ms p90) could not meet.
Productionized a fine-tuned SLM reranker at flat online latency, making a 44% NDCG@10 gain and 63% fewer bad results shippable.
Led the platform’s full lifecycle from an accepted architecture design review through production deployment, standing up 2 new repositories, service-to-service authentication, and 3 Production Readiness Reviews that standardized onboarding for new use cases.
Adopted genai-bench for inference benchmarking and contributed a bug fix back to the open source project.
Kept the ranking rollout healthy through A/B testing, resolving production issues in real time, implementing warmup orchestration to cut cold-start latency, and tuning KEDA downscaling to absorb cron-driven traffic spikes.
Contributed key design input to the LLM proxy integration, shaping the routing layer through which client teams consume the inference platform.
Spearheaded cost-saving initiatives delivering over $3.3M in annual savings, including sunsetting a legacy email alerts product (~$1.7M/year), cutting the team’s AWS account costs (~$1.44M/year), and removing unused data builders (~$180k/year).
Traced a GPU cost-attribution bug misreporting spend across every GPU deployment in the company, driving the cross-team investigation and proposing the fix.
Led the migration of bidding models onto the shared model-serving platform as the single point of contact for the initiative, establishing SLO monitoring and shadow testing for safe rollout, and consolidated bidding onto a global cluster, simplifying deployments and improving CPU utilization.
Drove the creation of comprehensive monitoring dashboards and SLOs for critical services, enhancing system observability and reliability.
Owned and resolved an 11-month availability failure in a business-critical data pipeline, coordinating 3 engineers and an intern across 5 partner teams, migrating 3 of its indexes to AWS, and restoring the model refresh cadence to daily.
Built the data-comparison tooling that validated those AWS index migrations within a 1% deviation, adopted by every team performing the migration.
Led the migration of SERP second-phase scoring off the index service onto the shared model-serving platform, from design through implementation, across three partner teams.
Led the handover of an entire product area to a new team, writing the transition documentation, running knowledge-sharing sessions, and onboarding their engineers into the on-call rotation.
Identified and drove company-wide storage savings, including disabling versioning on outbox S3 buckets (2.2PB reclaimed, ~$29k/month) and applying lifecycle policies to analytics buckets.
Pioneered research into Java 21 with Generational ZGC, documenting the findings and a benchmarking plan for adoption across the platform’s services.
Stepped up as team lead, mentored and onboarded numerous engineers and an intern, and supported the growth of 4+ engineers through regular 1:1s.
Co-created a new pair programming interview, building the questions and defining the scoring signals, removing 2 interviews from each candidate’s loop.
Set the team’s AI-assisted engineering practice, running demos and knowledge-sharing sessions on a spec-driven, agent-assisted workflow that other engineers adopted as their own.
Corrected the team’s testing strategy: wrote the proposal that became a team objective and guided a QA engineer in raising unit test coverage on a core service from under 10% to over 90%.
Presented the team’s technical direction and business impact to senior engineering leadership, and authored the runbooks, load-testing tools, and wiki documentation the team onboards from.
Acted as a key technical partner and point of contact for numerous cross-functional teams, unblocking critical initiatives and ensuring successful project delivery.
Improved developer productivity by enabling caching and incremental builds on a Gradle task used in every Java build company-wide, cutting time spent on it by 30%.
Feb 2017 – Oct 2020 · San Francisco, CA - Denver, CO - (Remote) Austin, TX
As a Lead Software Engineer on a travel platform, I focused on integrating and optimizing third-party connections to expand market reach and enhance system capabilities.
Led a team of four developers, providing technical guidance and overseeing project execution to success
Expanded market reach by integrating with external hotel, car, insurance, and flight systems in multiple countries
Architected and implemented a new system design to accelerate the integration of third-party APIs
Official NordVPN client in a Docker container; it makes routing traffic through the NordVPN network easy and secure with an integrated iptables kill switch
Presented the serving side of Indeed's small language model stack: platform capabilities, production cost and latency case studies, and guidance on when fine-tuning and self-hosting pay off over a third-party API.
SLM at Indeed (talk)
SciCon, Indeed's internal science conference · Dec 2025
Introduced self-hosted small language model inference at Indeed and the first production use cases running on it.
Julio is a strong leader, fantastic engineer, and wonderful teammate. As a teammate, he is always ready to support his colleagues in their immediate work and overall skill development. In addition, he is thoughtful of the collective experience of the team and is constantly looking for ways to improve processes. He is extremely collaborative, ensuring a synthesis of all related and interdependent elements of a project, especially those owned by other teams. He is thorough in his investigation and thoughtful in his recommendations.
Julio is an exceptional software engineer and an outstanding mentor. His technical knowledge and problem-solving skills are top-notch. He served as my mentor during my first software engineering internship. His guidance was invaluable, and has left a lasting impact on my growth as an engineer.
During my internship, I quickly saw how committed Julio is to improving team practices and engineering processes. He is responsive, efficient, and consistently leads team discussions with clarity and confidence.
Despite the demands of his role as a staff engineer, Julio always found time to mentor me in coding best practices, system design fundamentals, and—most importantly—how to frame my work within the broader goals of our team and the company.
Julio is an asset to any organization he joins!
I really appreciate Julio's contributions when it comes to evaluating and establishing new team processes. He's never afraid to speak his mind, even when he disagrees with the proposition being discussed, but he always does so in a way that facilitates conversation rather than hinders it, and as a result, the process always comes out stronger for it. He additionally encourages the team to find ways to measure processes, even if at first glance the process feels more qualitative than quantifiable.
