Sometimes the best way to understand a new thing is through analogy. Analogy: a comparison between two things, typically for the purpose of explanation or clarification. In this post we will explain the differences between a platform similar to Airflow, called Prefect, and SaasGlue
After raising over $43M in Series A and Series B, Prefect has generated a fair amount of buzz recently. Included in the ranks of investors are some pretty heavy hitters. Github, Valor Equity Partners, Kaggle, etc. Prefect positions itself as “The easiest way to automate your data”. There is no doubt they have a very useful product. Automation of software orchestration and data pipelines is a hot topic right now.
Prefect is a nice system. SaasGlue is better. Read on to find out why.
Prefect Hybrid Model Vs. SaasGlue Model
The Prefect Hybrid Model is built on a false premise. In a blog post from 2/10/2020, Prefect CEO Jeremiah Lowin cites an anecdote wherein an early investor claimed “There’s no way in hell I’m giving you our code or data’’. This led to Prefect’s Hybrid Model. Prefect does not have access to the client’s code. They create a metadata file based on the customer’s code and use it as a proxy for managing jobs.
The assertion that most customers won’t put their code and data in third-party clouds is just silly. Amazon AWS Lambda, Microsoft Azure Automation, and Google Cloud Compute Functions have thousands and thousands of paying customers who do just that. Cloud security is well-established with mainstream customers. Even highly sensitive customers like government entities are rapidly adopting cloud computing services. The FedRAMP program promises unparalleled security for government agencies and lists hundreds of cloud providers. https://marketplace.fedramp.gov/#!/products?sort=productName
Another case cited by Prefect to justify their hybrid model is HIPAA compliance. AWS, Azure, GCP, Oracle Cloud, and IBM Cloud are all HIPAA compliant vendors, and SaasGlue works with all of them.
The SaasGlue Model
SaasGlue code can be stored either in the SaasGlue cloud or on-premise (like the Prefect Hybrid Model). Why would you want the code to be in the cloud? One obvious reason is failover. Using SaasGlue, if one agent (system) is down, the job can run on another agent with the same tags. The Prefect Hybrid Model cannot do this.
Another benefit of using SaasGlue is convenient access to load balancing among ephemeral compute instances. This is similar to the distributed parallelized map reduce model in SPARK and Hadoop.
The SaasGlue model also makes cross-cloud orchestrations trivially easy. Prefect can’t orchestrate jobs that run on-prem, in GCP and in AWS without firewall exceptions. SaasGlue can.
Who runs that code anyway?
In the Prefect system, their agent actually executes the code. This is a common architecture for many automation platforms like Airflow. So you need a plugin for each language you want to run. Managing these plugins becomes a nightmare.
Note that the Prefect agent has full access to the code downloaded from the customer’s on-prem server. Tell me again how this Prefect system doesn’t have access to customer code?
In the SaasGlue system, the agent does not execute any code. It passes the job to local interpreters in the system. That way, there are no plugins to manage. Run whatever scripting language you like. SaasGlue can even orchestrate compiled code as an artifact. If the interpreter is not present in the system, you can download that as part of the job.
Multi Lingual!
Prefect makes you learn a new language. It looks a lot like Python. Python is a nice language, but there are lots of other nice languages out there as well. Like .net, Scala, Ruby, Powershell, Windows batch files, Node.js, etc. SaasGlue can easily automate workflows with any of these, or even a mix. SaasGlue even allows passing variable values between heterogeneous code execution environments and cross-cloud.
SaasGlue allows you to leverage the scripts and programs you already have. With Prefect, if it’s not Python, you’re out of luck. Rewrite
Secret Agents
There is a huge difference between the Prefect Agent and the SaasGlue Agent. The Prefect agent is for execution and orchestration. The SaasGlue agent is for connectivity to a local interpreter. The result is that the SaasGlue agent is super easy to deploy, while the Prefect agent requires extensive setup and configuration. SaasGlue has fully embraced light-touch, cloud-based orchestration.
Security
If your automation requires cross sub-net connectivity, the Prefect agent will require firewall exceptions. In contrast, SaasGlue uses OAUTH 2.0 standard security for agent and API access, and uses no inbound connections.
Types
Prefect offers 4 different kinds of agents, Local, Docker, Kubernetes, and AWS ECS.
SaasGlue offers 3 different agents as of today, Windows, Mac, and Linux. The Linux agent runs on any Linux machine, local, cloud, or ephemeral. SaasGlue also publishes a Docker image which can be used to deploy the agent. The SaasGlue agent has a very small footprint, and could easily be applied to IOT.
Another option with SaasGlue is agentless. If your data is accessible in AWS Lambda or Google Cloud Compute Functions or if your code is calling a public API, SaasGlue can leverage these agentless technologies. Look for a blog entry on this soon.
Where do you want your Orchestration?
Now we are getting down to the fundamental difference between SaasGlue and platforms like Prefect. Orchestration can happen in the cloud or on a local machine. If done in the cloud, like SaasGlue, then there are no boundaries to orchestration and automation. Provisioning is handled by the cloud infrastructure. Orchestration on a local machine dramatically limits your connectivity, leaving you configuring firewalls, managing plug-ins, being limited to scripting languages, and worrying about provisioning and administering the platform. We think that taking advantage of the massive benefits of cloud computing makes more sense. BTW, if you want to keep all your scripts on-prem, we can do that too. Trivially.
Originally published at http://blog.saasglue.com.
