Transwarp Sophon is a general AI platform to help enterprise users to quickly start building AI applications.For most enterprise users, the cost of creating AI platforms from scratch through their own power is huge, and requires a strong and powerful big data team and AI team as the basis. Based on big data platform and AI platform , Sophon enables business analysts and data analysts to easily build the corresponding AI models auto-matically or with the built-in industry templates, and thus to enhance business value.
Sophon has the following characteristics: one-stop graphical artificial intelligence development environment, modelling automation and recommendation, high-performance distributed machine learning algorithms, efficient interactive experience, perfect depth learning support and a large number of industry application templates.
• One-stop graphical artificial intelligence development environment. The whole process can be carried out by drag-and-drop operations including: data import, data exploration and preview, data preprocessing, feature engineering, algorithm selection, model training, model release, model management.
1. Data import:
Supports multiple data sources. Users can import data in a variety of ways, in addition to deep docking with Transwarp TDH, SQL databases, other Hadoop clusters, a variety of open source formats, and popular cloud storage formats.
2. Data visualization and exploration:
A wealth of visual display and statistical analysis. Users can visualize the data and analyze the data quality and data features to prepare for subsequent data preprocessing and feature engineering.
3. Data preprocessing:
perfect and diverse pre-processing function. Sophon provides up to 50 kinds of operators, including but not limited to data cleaning, data protocol, data conversion, etc.
4. Feature engineering:
Flexible feature extraction methods. Sophon provides a variety of feature extraction and feature conversion methods.
5. Algorithm selection:
High-performance distributed algorithms. Users can choose local or distributed algorithms, including common classification, regression, clustering, recommendation, time analysis, statistics and other machine learning algorithms, such as a variety of classical neural networks, as well as NLP and image-related. Meanwhile, Sophon provides streaming machine learning by combining with the SQL streaming engine Transwarp Slipstream.
6. Model Training:
Efficient training methods and multiple model evaluation methods. Sophon provides a variety of low-level operating platforms, through Kubernetes+docker mode for resource control and scheduling. For the training model, the user can evaluate the model by different indexes and models.
7. Model release and Model Management:
powerful model publishing and management functions. Users can choose a satisfactory model to publish, either by exporting the model in PMML format, or by deploying its API services on-line, or by scheduling the workflow in a regular manner. A well trained model can be versionized.
• Modelling automation and recommendation. Sophon provides users with one-click modelling automation, and modelling recommendation. One-click modelling automation enables users to do data type detection, data preprocessing, feature selection, model selection and parameter tuning automatically, by utilizing the built-in intelligent algorithms. Users can achieve a promising precision without any manual work. Users can also use the modelling recommendation to achieve a higher precision. The modelling recommendation is built upon data and fine-tuned models accumulated from different industries. Based on the deep understanding with the current scenario, Sophon will recommend several well-fit operators for users to choose from, thus to achieve better performance by mixing automation and manual tuning together.
• High-performance distributed machine learning algorithms. Support more than 100 kinds of distributed algorithms. For common machine learning algorithm, the performance in Sophon is relatively 3 to 10 times times faster. Sophon also supports streaming prediction for around 10 common algorithms, based on the indus- try's most powerful SQL streaming engine, “Transwarp Slipstream”, and thus can support streaming machinelearning. In addition, users can easily write their own algorithms through the encapsulation of custom operators in languages such as Python and Scala.
• An efficient interactive experience. Through Sophon, the user can quickly carry on the data exploration, the iterative modelling process, greatly shortens the modelling cycle. The model can be tested through A/B test in the production environment more timely. Thanks to the ease use of Sophon, the company can help with a unified artificial intelligence platform, providing enterprise-class support including: multi-tenancy, authentication, security control, team collaboration, model sharing, distributed scheduling, and so on.
• Perfect deep learning support. Users can construct complex neural networks directly by dragging and dropping. The structure of a good neural network hierarchy is very easy to understand. Sophon is built upon “Hubble”, as the unified algorithm framework, which can provide Tensorflow and MXNet framework at the same time, ensuring seamless connection with complex preprocessing operators. In addition, Sophon is optimized ondistributed GPU algorithm, which has double performance improvement. Sophon also incorporates a wide range of classic neural network structures, including a “Deep and Wide” model that combines depth learning with wide learning.
• A large number of industry templates. In the Common machine learning field, Sophon integrates such templates as securities time-series analysis, credit card staging, financial product recommendation, precision marketing and customer churn warning. For NLP and image two deeplearning areas, Sophon also integrates common solutions. In the field of NLP, Sophon supports new word identification for news, named entity recognition, semantic search of massive data, public opinion monitoring, enterprise text risk management, question answering system and so on. In the field of image, Sophon supports face recognition, license plate recognition, image classification, image search, object detection and so on.
Sophon has eight major technical features.
• Ease of use. Through the one-stop graphical interface, data analysts and business analysts can quickly start machine learning. Even for senior data scientists, an efficient interactive experience shortens the cycle of model tuning.
• Intelligence.Based on the accumulated models from different industries, Sophon itself is intelligent enough to ease the difficulty of modelling through modelling automation and modelling recommendation.
• Multiple data source access. The user's underlying platform can choose to use Transwarp TDH, as well as other data sources. After the data source is connected, it will unify the view and specification, to help user to focus on modeling, without wasting the time for data integration between different data platforms.
• Platform openness. The user can easily extend the algorithm through Python or Scala. And for the integration of other technologies or frameworks, Sophon provides a common algorithm framework layer and algorithm interface layer that users can easily access.
• Graphical visualization. Sophon provide graphical interfaces for the whole ML pipeline: drag-and-drop of operators, data preview, model tuning and model management.
• Performance. Sophon has up to 3 to 10 times performance optimization for open source algorithms, with additional parallel optimization for distributed GPU.
• Enterprise-class features. Compared to individual users, enterprise users will care more about whether the product can support multi-tenancy, authentication and security control, team collaboration, model sharing, resource management and so on. And Sophon can solve these problems excellently.
• Precanned Industry templates. For most users, the precanned industry templates are critical to help them start business quickly without requiring extra training and learning.
Based on the accumulation of machine learning experience in multiple industries, Sophon can support scenarios include but are not limited to:
credit risk, customer life value, business circle analysis, user portrait, intelligent investment, risk control, quantitative investments, real-time recommendation, loss warning, Q&A system, semantic search, knowledge map, image recognition
Spam message detection, package recommendation, intelligent network optimization, user behavior analysis, precision marketing, user churn analysis
user behavior analysis, user portrait, product recommendation
production equipment fault detection, intelligent reliability maintenance, equipment monitoring
Traffic energy industry:
license plate recognition, image recognition, object tracking, equipment monitoring
license plate recognition, image recognition, object tracking, equipment monitoring Government agencies: Text mining, public opinion analysis, new word discovery