Infrastructure for AI Agent and Application Development: Deploying a Private AI Server with Proxmox VE
In today’s fast-paced world, artificial intelligence is driving innovation across industries, and the success of AI projects depends on robust, high-performance infrastructure. A development team from Denmark needed a hardware platform capable of supporting AI analytics applications powered by the YOLO neural network.
Recognizing the project’s strong business potential, the INTROSERV team quickly stepped in to design, configure, and optimize the platform according to the client’s technical requirements. The result was a fully controlled, secure, and high-performance environment that allows the team to focus on developing cutting-edge AI solutions efficiently.
Goals:
Preparing a Server Platform for YOLO-Based Software Development
- Selecting an appropriate platform Choose a server configuration with GPUs within an acceptable budget to support interaction with the YOLO neural network.
- Virtualization platform The software development required multiple virtual machines for different team divisions.
- Secure connection between the data center and the development office To protect confidential information, a fully encrypted data channel was necessary.
- Disk subsystem redundancy Although the project did not require real-time object recognition, prolonged downtime due to host reinstallation or reconfiguration was highly undesirable.
- Backup and fast recovery in case of failure The project involved numerous VM configurations, so a reliable recovery source was essential for rapid platform restoration.
- Stable network connection The data source for application development consisted of video and large volumes of photos. It was crucial to support multiple parallel video streams while leaving sufficient bandwidth for smooth remote work.
Results:
Deployment of a GPU server with Proxmox VE and secure development environment
- Platform selection A dedicated server with two NVIDIA RTX 2080Ti GPUs was installed, providing sufficient performance for processing video and images with the YOLO neural network.
- Virtualization with Proxmox VE An open-source hypervisor was deployed to manage VMs, allowing GPU passthrough to virtual machines.
- VPN server and tunnel setup All traffic between the server and the development office is routed exclusively through a WireGuard VPN tunnel.
- RAID1 configuration for all storage groups Single points of failure from disk malfunctions were mitigated. Proactive monitoring with anomaly alerts was implemented.
- Backup service for rapid recovery Regular backups of all virtual machines were configured using Proxmox Backup Server.
- Stable network connection An upgraded bandwidth plan was selected to ensure smooth data transfer and uninterrupted remote work.
Context: How Computer Vision and the YOLO Neural Network Support New Software Development
A team of developers specializing in AI-driven business solutions approached INTROSERV with a request for an infrastructure capable of running neural network inference, specifically for computer vision tasks, including Ultralytics YOLO models.
Their goal was to leverage computer vision neural networks to create business analytics applications. Among their current developments, several stand out for their practical value:
- A compact security monitoring system
- A construction site equipment tracking system
- An automated warehouse inventory module
The client planned to build a flexible and convenient development platform that could be adjusted according to the requirements of each new project.
They also expected expert support for server administration in case of issues during software component installation.
Privacy and data protection were key priorities, which is why they chose dedicated hosting over shared or cloud-based alternatives.
This project required not just “a rented server,” but a specialized, secure environment designed to handle the expected workloads.
The Challenge: Integrating Neural Networks with Business Process Automation Software
The primary challenge was to develop analytics software and seamlessly integrate it with neural networks.
An additional complexity came from training YOLO models on specific datasets to improve the software’s accuracy in counting and recognition tasks. The client’s development team took responsibility for solving this aspect.
Infrastructure preparation, however, was delegated to us. From the start, the project captured the interest of INTROSERV’s technical specialists. Thanks to clear goals, a well-defined technical specification, and the availability of the right hardware configuration, we quickly found an optimal solution.
We knew exactly how to handle large data transfers to the data center, including high-resolution test video streams while maintaining 100% security.
The Solution: Dedicated GPU Servers with Proxmox for YOLO-Based Analytics
The INTROSERV team designed and implemented a multi-component solution using two dedicated servers, leveraging the Proxmox VE virtualization platform and Proxmox Backup Server. This setup allowed for efficient resource allocation, task isolation, and secure data protection through full VM backups.
The entire infrastructure was built on multiple virtual machines:
Virtual Machine #1: Network Gateway based on pFSense
- Purpose: Establish a secure communication channel and control access to the server infrastructure.
- Implementation: INTROSERV system administrators deployed a pFSense software router on this VM and configured a WireGuard VPN server. The VPN server connected the office internal network with the data center server through a secure tunnel. All traffic within the VPN tunnel is encrypted, ensuring confidentiality and data protection during transmission.
Virtual Machine #2: "AI Hub" of the System (YOLO Neural Network + GPU)
- Purpose: Fine-tune the model using reference frames from the video surveillance system of each specific site and perform object recognition on incoming video streams and images.
- Implementation: This is the key virtual machine. Using PCI Passthrough technology in Proxmox VE, the NVIDIA GeForce RTX 2080 Ti GPU was passed directly to this VM. This provided near-native performance required for two main tasks:
1. Model Training: Fine-tuning the YOLO model on reference frames from surveillance cameras and photo archives to improve accuracy.
2. Recognition: Continuous processing of video streams, object detection, counting, and grouping according to the application’s requirements.
Two such VMs were set up to facilitate parallel development and improve workflow efficiency for the developer team.
Virtual Machine No. 3: Analytics and UI
Tasks: Test environment for running applications at different stages of their development
Implementation: Linux OS was deployed on this VM and the basic dependencies specified in the technical specifications were installed, and an isolated network connection with other VMs was configured.
Proxmox Backup Server:
Tasks: storage of virtual machine backups
Implementation: To accomplish this task, INTROSERV technical specialists chose a solution based on Proxmox Backup Server. A specialized solution from the developers of Proxmox VE, which logically complements the ecosystem of the popular virtualization solution. PBS was installed on a separate server connected to the main server via a separate network interface.
Dedicated Server Configurations Used
YOLO server
- Platform: Asus ESC4000 G4
- Location: Netherlands
- CPU: 2x Intel Xeon Silver 4208 16 Core 32 Threads 2.1/3.0GHz
- Memory: 64GB REG ECC DDR4
- Disk: 2x 500GB SSD Hardware RAID 1
- Disk: 2x 3.84TB NVMe Software RAID 1
- GPU: 2x Gigabyte GeForce RTX 2080 Ti 11GB
- Port/Traffic: 2 Gbit/s - Unmetered
- VLAN: 10 Gbps VLAN
- AntiDDoS: DDOS 20Gbit/s protection
- Operating system: Proxmox VE 8
PBS
- Platform: Dell PowerEdge R240
- Location: Netherlands
- CPU: Intel Xeon E-2236 6 Core 12 Threads 3.4/4.8GHz
- Memory: 16GB ECC DDR4
- Disk: 2x 250GB SSD Hardware RAID 1
- Disk: 4x 16TB SATA Hardware RAID 10
- Port/Traffic: 1 Gbps - 50TB Traffic
- VLAN: 10 Gbps VLAN
- IP-KVM: iDRAC
- AntiDDoS: DDOS 20Gbit/s protection
- Operating system: Proxmox Backup Server
Partnership with INTROSERV Accelerating Innovation
By entrusting all infrastructure tasks to INTROSERV, the development team was able to fully focus on what matters most – improving their AI model and working with data.
Instead of spending time selecting hardware, configuring security, and ensuring fault tolerance, the client received a ready-to-use hardware environment that can be applied repeatedly to each new project. INTROSERV took care of the entire technical side, providing not just equipment but a reliable foundation for productive work. At the time of publication, the client had already begun the neural network training stage and reported excellent performance from the proposed solution within the first few days. This demonstrates that INTROSERV is not just a service provider but a true technology partner capable of ensuring success in complex IT projects.
For assistance in selecting the optimal infrastructure for your project, please contact the INTROSERV's sales department.
