Ultron Documentation

What is a programmable logic controller (PLC)?

A programmable logic controller(PLC) is a simplified computer that does not have a screen, keyboard, and many other peripherals found in a typical PC. It is a computer-controlled industrial system that continuously monitors the state of input devices and makes decisions based upon a set of specific rules and input through a custom program to control the state of output devices.

In an industrial setting, the PLC is housed within a control panel and is connected through Ethernet or USB to a laptop or desktop computer that has a SCADA application installed and is used to program the PLC. However, depending on the inputs and outputs, a PLC can monitor and record run-time data such as machine productivity or operating temperature, automatically start and stop processes, generate alarms and alerts if a machine malfunctions, and much more. They can be found in a variety of industries and everyday products such as washing machines, HVAC systems, traffic lights, and elevators. Depending on the manufacturer, PLCs can be programmed in a variety of programming languages. For example, Ladder Diagram, Structured Text, Functional Block Diagram, Instruction List, or Sequential Function Chart.

Key components that set PLCs apart from industrial PCs, microcontrollers, and other industrial control solutions:

  • CPU: serves as the PLC's brain, performing logic and arithmetic operations and updating the inputs and outputs.

  • I/O modules: can be analog or digital and provide information to the CPU and trigger specific results. Sensors, switches, and meters are examples of input devices, while relays, lights, valves, and drives are examples of output devices.

  • Communications: in addition to input and output devices, a supervisory control and data acquisition (SCADA) system is typically used to record data from the PLC and monitor and control multiple connected devices.

  • HMI (Human Machine Interface): To interact with the PLC in real time, users need an HMI. These operator interfaces can be simple displays with a text readout and keypad, or large touchscreen panels resembling consumer electronics, but they all allow users to review and input data to the PLC in real time.

Ultron is a NVIDIA®️ Jetson-based Programmable Logic Controller (PLC) that brings high computing power to the edge for industrial and commercial autonomous infrastructure developments and deployments. Based around NVIDIA: registered: Jetson™️ modules, Ultron allows users to run computationally complex AI workloads and process data from high-resolution sensors whether it’s to be deployed in industrial environments, at road intersections or for a home automation project

With Ultron, we want to change the development and deployment of automation and autonomous infrastructure with a state-of-the-art compute and sensor fusion platform that goes beyond the capabilities of traditional industry-grade PLCs.

Introduction

Ultron is a PLC system based on NVIDIA® Jetson™ that brings high computing power to the edge for industrial and commercial applications requiring logic-based decisions for automated processes and machines. It comes with three GPU variants, TX2 NX and Xavier NX, and Orin NX. Ultron delivers up to 100 TOPS for modern AI workloads and data processing from high-resolution sensors. Ultron is designed with a DIN-rail mounted and flexible system structure that allows for the expansion of up to seven I/O units under a single main unit. It is suitable for the building of smart factories, traffic signal controls, and smart buildings.

Key hardware features

  • Powered by NVIDIA®️ Jetson™️ modules

  • Dual Ethernet ports (one GigE and one optional 2.5G LAN port)

  • Mini DP port and a built-in OLED for HMI display

  • Built-in 128G NVMe storage

  • Optionally supports 5G/LTE/Wi-Fi features

  • Supports various input/output connections

Key hardware features image

Modular design

Main unit

The main unit is powered by the NVIDIA®️ Jetson™️ module, which has three GPU variants, including Jetson Orin NX, Xavier NX, and TX2 NX, and is equipped with two RJ45 ports, allowing for the use of expandable IP cameras for advanced video detection and analytics. Ultron's main unit also supports 5G, LTE, and Wi-Fi wireless alternatives for comprehensive network building, increasing the seamless communication for advanced programmable logic control. Furthermore, the main unit includes 1xI2C and 1xCan Bus, both with RS232 and RS485 configuration, giving the developer options for realizing the application's control requirements. Last but not least, Ultron is an out-of-band (OOB)management device with backup connectivity to the application to overcome network outages and disruptions. In addition to the security of the device, it includes an inertial measurement unit (IMU) sensor, which prevents unsecure device movement and raises the safety to ensure the smooth operation of the system.

Combined with the main unit and I/O unit, Ultron can accommodate up to seven additional I/O blocks for highly dynamic application scenarios. Considering the scalability of application scenarios, Ultron is designed to handle multiple control systems and address various environmental requirements, making it easier for developers to address future challenges.

Key hardware features modular unit

Processor

SoM

- Xavier NX/ TX2 NX - Compatible with Orin NX

Inputs & Outputs

Display

- 1x OLED

- 1x mini DP

Ethernet

- 1x RJ45 GBE (10/100/1000M)

- 1x RJ45 2.5G (optional) [1][3]

USB

- 1x USB3.1 Type A

Serial Port

- 2x RS232 or RS485

Communication

- 1x I2C

- 1x CAN bus [2]

DIP Switch

- 2x DIP switch (4 pins) for RS232/RS485 configuration

LED indicator

- 1x Power Status

- 4x LAN Status (2 LEDs for each LAN port, "Active & Link")

- 2x Serial ports (active)

- 1x CAN bus (active)

Button

- 1x Reset

SIM Slot

- Nano SIM slot

Storage

Storage

- Default with 128G NVMe SSD

Expansion

Optional SKUs

- WWAN and GNSS: LTE / 5G

- Wireless: Wifi [3][4]

Inputs & Outputs

Power Input

- 24V DC in

Connector Type

- 2-Pin w/ lock Terminal block

Inputs & Outputs

Dimension

- 100mm * 76mm * 112.5mm

Installation

- DIN Rail

Inputs & Outputs

Humidity

- 5 ∼ 95% RH (non-condensing)

Temperature

- Operating Temperature: -25°C ∼ 60°C

- Storage Temperature: -40°C ∼ 85°C

Regulation

- RoHS

Certification

- CE/FCC

Inputs & Outputs

BSP Version

- Jetpack 4.6.1

I/O unit

I/O unit image

The I/O unit can be expanded to seven blocks for a flexible design that can address the application's challenges. Single I/O units have 4x analog inputs, 8x digital isolated inputs, 8x digital isolated output relays, and 4x digital outputs, allowing developers to build applications for traffic light control, factory automation, and various sensors.

LED Indicators

Power

- 1x Power status

GPIO

- 1x GPIO_INT

ADC

- 1x ADC_INT

- 1x ADC_CNV

I/O status

- 24x I/O status

Inputs & Outputs

Analog

- 4x Analog input, 0∼10V

Digital

- 1x CAN bus [2]

Relay

- 8x Digital isolated output relay, AC 250V/5A

Digital

- 4x Digital isolated phototransistor output, 5∼24V

Address Setting

ADC

- 5 bits on DIP switch

GPIO

- 3 bits on DIP switch

Expansion

- Board to Board connector for I/O unit

Power

Power Input

- 24V DC through board to board connector from Main Unit

Mechanical

Dimension

- 100mm * 76mm * 112.5mm

Installation

- DIN Rail

Key software features

Ultron’s software-defined architecture enables the application of software logic through a centralized interface, allowing for greater flexibility in the development and deployment of autonomous infrastructure. The key features of the software include:

On-device data acquisition and control software

  • User authentication to prevent unauthorized access

  • Automatic registration and display of physically connected modules on the UI

  • Allow real-time querying and status monitoring through visual representation of control blocks with feedback mechanism on the user interface

  • Save the current configuration or load a previously saved configuration.

  • Emulate analog and digital I/O and provide access to logic elements such as timers, counters, and control bits to meet your needs

  • View deployed software configuration, communication settings, general information, and firmware versions of the device

  • Modify parameter values in the parameter editor to control local or remote I/O

  • View health metrics such as GPU, CPU performance, and average temperature of the compute unit

Flow-based visual programming tool

  • A low-code integrated development environment for implementing logic on PLCs and automating processes

  • A drag-and-drop interface for controlling inputs and outputs without having to know about underlying programming concepts

  • Implement custom nodes to interact with Ultron’s analog and digital GPIO

  • Wire automation flow easily using a wide range of communications protocols and function nodes

Gstreamer plugin and vision SDK

  • Add and configure RTSP streams using the UI

  • Integrate NVIDIA DeepStream and Gstreamer video analytics elements and control output signals based on inference result using customized SDKs

  • Simplify the process of vision system integration by removing the need for specialized tools or programming languages to apply image processing logic

  • Deploy and concurrently run multiple AI models in each module using the modular software design

Visualization and HMI

  • Create easy to understand visualization and interactive elements that are mapped to actual values and controls

  • Display real-time data and visualize inference results on a single centralized dashboard

  • Allow operator to monitor and interact with process control systems to improve production efficiency

  • Automate and display visual alarms on the HMI to ensure safety and security

System Architecture

System Architecture Image
System Architecture Image mobile version

Data collection and system supervision

Ultron is powered by NVIDIA Jetson Linux4Tegra (L4T), a Linux-based embedded operating system developed by NVIDIA for all Jetson platforms and supported by NVIDIA Jetpack SDK, a comprehensive set of libraries for acceleration of GPU computing, multimedia, graphics, and computer vision. Benefits of using Linux as a platform include background logging of critical operations and support for open communication standards such as SSH for secure communications.

A centralized supervisory software that monitors and controls the entire platform sits on top of the Jetson-based PLC hardware. The software processes collected data from inputs and sends commands to control output processes. Because the device's software is cloud-native, the containerized app can be easily managed and deployed at scale.

GPU-accelerated computing with Ultron

The collected data is useful for AI model training and for developing seamless streaming pipelines to extract meaningful real-time insights. The NVIDIA DeepStream SDK is a comprehensive analytics toolkit for AI-based multi-sensor processing. The toolkit is based on the GStreamer multimedia framework and includes a GPU-accelerated plug-in pipeline for building end-to-end AI-powered applications that analyze video and sensor data from connected inputs.. By leveraging Jetson’s high computing power and the unified SDK, , DeepStream and GStreamer elements can be easily integrated into the automation system and shorten the time required to build and deploy real-time Intelligent Video Analytics (IVA) applications. For example, engineers can build end-to-end DeepStream pipelines to quickly convert raw video input data into insightful annotated video output.

AI Inference at the edge

Aside from increased computation capability, real-time inference at the edge reduces latency significantly. This is vital when connectivity is unavailable, such as with remote devices, or when the latency to send data to and from a data center is too long. Edge AI minimizes data transfer between edge devices and their data centers. There is also more privacy and security by storing the data locally.

Visualize and derive insights from models' output

A human-machine interface (HMI) is an interface that enables humans to monitor and control a device. An HMI is required in a control system and is usually connected to a single PLC or process to enable operators to change the flow set point and enable alarm conditions in the event of a loss of flow, high temperature, or anomalies, and the condition is displayed and recorded. The following figure shows a customized HMI connected to Ultron that streams the AI model’s output, visualizes important device metrics, and includes interactive elements for I/O controls

Ultron UI Image

Edge Management

Deploying and managing a large network of AI infrastructure at the edge remains a challenge for organizations especially when the edge devices are spread across geographically dispersed locations. From remote debugging, software management and maintenance to system monitoring, a centralized management suite like FleetTrackr provides a simplified and secured method to provision, manage, maintain, monitor, and update thousands of appliances, entirely over-the-air. For example, Ultron could be widely distributed across vast distances when deployed for industrial automation and FleetTrackr can be used to streamline management.

There are six key components in FleetTrackr provisioned by the FleetTrackr Management Suite:

  • Order Management: Keep track of order status and manage device inventory

  • Device Management: Easily monitor device metrics and device status, get access to crucial device performance KPIs based on historical data. Access device documents, specification sheets, and user manuals for ease of deployment.

  • Container Management: Update or restart your container with a single click and create a group of devices to provision containers easily.

  • Firmware-Over-The-Air: Firmware update, backup and recovery through remote access

  • Issue Management: Group edge devices and raise tickets when a device goes faulty, create and manage site leaders and teams to resolve tickets, schedule regular device maintenance tasks, and get access to KPIs on ticket history.

  • Predictive Maintenance: Detect hardware/software anomalies through automated anomaly detection and classification, predict time-to-failure

In short, FleetTrackr is readily available to be used for deploying and managing Ultron at scale. Instead of spending weeks planning and executing deployment plans, administrators can upgrade AI solutions, manage applications, update system firmware and software, streamline operations and administrative tasks, and monitor the health metrics of the fleets from a single management panel.

FAQs

Which of the computing platforms is Ultron  capable of using?

Ultron can be configured to use any of NVIDIA’s Jetson SoCs available in the SODIMM form factor, including the JetsonNano™️, TX2 NX™️, and Xavier™️ NX.

What is the difference between the three configurations?

Jetson Nano™️ has the lowest CPU and GPU cores, making it ideal for low-power environments. TX2 NX increases CPU and GPU capabilities, while Xavier NX offers even better performance while also increasing RAM size, making it ideal for running demanding deep learning models and more complex applications. The same software suite is available on all Ultron computing platforms.

What are the benefits of having an embedded GPU?

This enables you to run high-performance machine learning and deep learning models directly on Ultron. This can be used in conjunction with cameras and sensors to create complex rule sets for your applications.

What is the storage capacity?

Ultron includes a 128GB NVMe SSD as standard.

Which operating system does Ultron use?

Ultron's operating system is based on Ubuntu 18.04 LTS.

Can I access Ultron remotely?

Yes. Ultron supports SSH-based remote login. Ultron can be connected to your network through its Ethernet ports or optional WiFi and LTE/5G modules.

How can I control the digital I/Os and relays

As these peripherals are connected to the embedded Jetson device, you can control the IOs and relays using the control software preloaded in the Jetson module and programmed using the integrated visual programming tool.

What other communication protocols are available?

Aside from Ethernet and USB, Ultron has two serial ports that can be configured to use the RS485 or RS232 standards. Ultron also includes I2C and CAN buses for connecting to digital sensors that use these communication protocols

How can I configure the ADC and GPIOs?

The IO expansion board includes two sets of switches for configuring the GPIO ports and the ADC. For more information about all configuration parameters, see the user’s manual.

Troubleshooting

The device is not powering on

Ensure that the 24V input supply is firmly and securely inserted into the appropriate input connector. When power is applied, the PWR LED switches on and Ultron boots up automatically.

There is no output when I connect the display

Ultron uses a miniDP connector as its display output. You can connect to a display with a DisplayPort input. To connect to an HDMI display, use an active DisplayPort to HDMI converter.

I cannot access the device over Ethernet

Ensure that Ultron is powered on and that an Ethernet cable is securely connected to the LAN0 or LAN1 ports. The link and activity lights flashes when the connection is successful.

I cannot read the digital sensor’s output

Ultron's digital inputs accept voltages ranging from 5-24V. Each input’s corresponding status LED lights up whenever it is pulled HIGH.

The ADC and GPIOs are not working correctly

Ensure that the configuration switches have been properly configured.

Use Cases

System Architecture Image
System Architecture Image

INDUSTRIAL AUTOMATION FOR SMARTER
FACTORIES

To date, a large portion of industrial automation is based on logic-based control from PLCs for a variety of processes and systems. The fundamental role of a PLC is to automate processes by sending programmed control functions computed from signals received from connected inputs such as a sensor, switch, thermometer, or relay to output devices. This system architecture, also known as a supervisory control and data acquisition (SCADA) system, is a secure and standardized automation infrastructure for sharing diagnostics and analytics. For example, PLCs are often used in industrial machine vision and automation systems for image processing and quality inspection, in pick-and-place operation of autonomous forklifts and robotic arms in manufacturing environments, as well as in downstream to upstream process control in oil and gas plants.

PLC implementation in action

PLCs have traditionally been widely used due to their dependability and robustness, as they require minimal intervention to operate. Optical quality inspection using machine vision systems is one of the most widely used methods for ensuring quality control in high-precision manufacturing. In production, for example, machine vision systems use sensors (cameras), processing hardware and software algorithms to automate complex or mundane optical inspection tasks and precisely guide handling equipment during product assembly.

Bringing deep learning to the factory floor

With the emergence of Industrial Internet-of-Thing (IIoT), Artificial Intelligence-of-Thing (AIoT) and robotics, manufacturers are developing smart factory strategies that use AI on automated manufacturing processes in production environments, freeing up people to focus on handling exceptions and making higher-level decisions in order to remain competitive in the industry. For example, to achieve higher detection accuracy, manufacturers are beginning to use deep learning technologies and real-time computer vision applications that can be performed at the edge. Traditional inspection tasks use rules-based machine vision solutions for part inspection and leak detection, but deep learning approaches such as convolutional neural networks (CNNs) can help to optimize inspection accuracy without any impact on production.

System Architecture Image

Ultron for smarter factories

To address this, SmartCow designed an all-in-one control platform that combines a wide range of industrial functionalities such as machine vision, PLC, AIoT, and robotics, complemented by a control software. This control software integrates I/O control, computer vision, and video analytics by providing the necessary vision and PLC capabilities in the form of function libraries, I/O blocks, and APIs that can be called up from Ultron, all configurable through a browser-based programming tool, allowing engineers to program in modern programming languages such as JavaScript and Python. Furthermore, NVIDIA hardware-accelerated SDKs such as the Isaac SDK for robotics algorithm development and the Deepstream SDK for streaming video analytics can be used to create high-performance AI applications. In terms of connectivity, the dual ethernet ports, 5G/LTE, Wifi, CANBus, and I2C options make Ultron an ideal PLC for multiple protocols and communication support.

Conclusion

To summarize, Ultron is built to improve industrial automation processes that lack the agility, programmer productivity, and development scalability needed for effectively interacting with sophisticated next-generation AI computing workloads that are often deployed on the cloud. We discovered that advanced computing capabilities are in high demand at the industrial edge control system. Nonetheless, SmartCow's approach of using Jetson-based programmable logic controllers to easily integrate edge AI with flexible industrial automation technology is applicable to a wide range of industries beyond factory automation, and it is the company's goal to innovate next generation solutions toward industry 4.0.

System Architecture Image
System Architecture Image

SMART TRAFFIC: SIGNAL CONTROL AT INTERSECTIONS

With the launch of CityStation, an AIoT platform for environmental data acquisition and monitoring, SmartCow is one step closer to boosting smart cities initiatives with weather data-driven services. While our team was working on a Jetson-based PLC for industrial automation, we discovered the potential of Ultron's to be deployed at traffic intersections alongside CityStation for smart traffic signal control.

Building a smart traffic signal control module

A traffic signal box has two main components: a controller and a conflict monitor. The controller within the traffic signal box serves as the traffic signal's brain. Depending on the intersection's phases (4 or 8 phases), it takes input from proximity sensors and loop detectors installed beneath the road or from video streams from birds' eye view cameras overseeing the intersection, which uses pixelation changes to estimate traffic flows. After that, the controller is programmed with vehicle movements, lanes, and timing sequences to control traffic flow and adjust traffic signals as needed. The conflict monitor, on the other hand, serves as the traffic signal control system's security guard. When it detects malfunction signals in conflicting directions, it shuts down the signals and returns to blinking red / yellow. When a conflict is detected, it exchanges data packets with the controller and turns the signal lights at the intersection on flush. It also monitors the controller's power voltage.

The problem with conventional traffic signal control

Although this conventional method of calculating traffic signal timing is more reliable and less prone to error, it does have some drawbacks. While the timing controller fixes the time sequence for every phase, it does not adapt to the intersection's real-time traffic. For example, the road most vehicles use to get to work in the morning may be more congested in the morning but less traveled in the afternoon, and it is usually the opposite direction that is congested. Furthermore, the installation of proximity sensors and loop detectors beneath the road is costly and resource intensive.

Ultron for adaptive traffic signal control

To solve this problem, SmartCow's AI engineering team proposes using Ultron as an adaptive traffic light controller based on computer vision AI. The signal system adapts to the current traffic situation by performing inference on live feed from a fusion of AI sensors and machine vision cameras installed at traffic junctions to calculate real-time traffic density by detecting and classifying vehicles at the intersections. The vehicle detection model calculates traffic density at each phase, and the algorithm sends the scheduling timing to the controller along with the optimized green and red signal duration. Aside from traffic density, the model considers the number of lanes on the road, vehicle types, and buffer time for vehicles to restart after stopping. A conflict monitor is also added to the PLC module that ensures that road safety is prioritized. Urban engineers can also view data using SmartCow's RoadMaster, an AI-enabled traffic management system for real-time data analysis that enables capabilities such as traffic anomaly and violation detection, turn counts, and lane utilization analysis.

An adaptive traffic signal management system not only reduces traffic congestion on the road, but it also contributes to lower CO2 emissions and shorter travel times for road users. Furthermore, adaptive traffic signal control relies less on loop detectors and proximity sensors that are often expensive and not robust to withstand harsh weather conditions.

System Architecture Image
System Architecture Image

SMART HOME: HOME AUTOMATION AND ACCESS CONTROL

A smart home can be achieved by using a home automation system that integrates smart technologies such as IoT, sensors, electrical appliances, and mobile devices, allowing homeowners to monitor and control their home. A smart home project is easy to begin with. In fact, many homeowners began their home automation project by purchasing a small single-board computer, such as the Raspberry Pi, and open source programming tools. Another option for a control module for smarthome systems is a PLC, which is commonly used for controlling electromechanical processes. There is no end to the scenarios that a smart home can achieve.

Here are some examples:

  • Access controlled doors, gates, and blinds

  • Remote and intelligent control of air conditioning systems

  • Adjustment of lighting intensity based on user preferences

  • Smart light switching in the mornings and at night

  • Heating water bathing and making tea

  • Temperature and pressure control in a swimming pool or jacuzzi

  • Controls for pump motors used in garden irrigation and water fountains

  • Suspicious activity detection in the home, which automatically activates the alarm system and sends alert messages

Ultron, which is based on NVIDIA® Jetson™platforms, is an excellent choice for the aforementioned use cases. It includes control software that is simple to use and requires minimal programming knowledge or integration with other software. Whether or not you are familiar with NVIDIA DeepStream SDK, a streaming analytics toolkit for AI-based multi-sensor processing video or image, is an excellent tool for AI application development. NVIDIA DeepStream SDK supports multiple frameworks and extensive reference of pretrained models. The DeepStream toolkit is readily available for use with Ultron's I/O modules, allowing you to apply programmable logic based on sensor readings and video inference results and send appropriate output signals to control the devices.