Tag Archives: scripting
The Problem (0:00)
Today we’re going to go over Swift scripting. At Venmo, we work with designers every day. One thing that we collaborate on is asset handoffs. Designers might update an icon, and so they somehow have to get it to us. Our process used to follow this sort of pattern. The designer would design the assets, export them, and upload them to Dropbox or send them over Slack. Sometimes they would email or text it to us – however they wanted, they would just get it to us. The engineer would add it to the venmo-ios codebase, which is our main repo.
This wasn’t very good. It was a simple process, but there were too many links and messages flying around. One designer might email an engineer, who would share it with another developer who was working on a feature that used the same asset. It became chaotic. Another thing that might happen was when the asset changes. It might get updated, but be cropped incorrectly. You wouldn’t notice that it changed until you actually saw the design on the device. Then you would have to email or Slack message the designer to ask for a new version. That’s also not fun.
Then, there was no easy way for the designer to see all the assets in one place. Before, they would have to go into our Xcode project or Github repo and then click through many folder to finally find everything in the resources directory.
The Solution (1:54)
Then we had an idea. Why not pull this out into CocoaPods? We already had our venmo-ios repo, so let’s make venmo-ios-images for the assets. Our new process looked like this. The designer would again design and export the assets. Then, the designer would make a pull request to the new repo. Our designers are really awesome about that, and sometimes even show us screenshots in the pull request. The engineer then reviews and the changes and merges. We generate our
UIImage category for the new images, so we don’t have stringly typed images lying around everywhere in our codebase. All we have to do then is
Script All the Things! (3:03)
Our solution seemed pretty simple, but what about the step where engineers generated the
UIImagecategory for the new images? What do we mean by generate? Does that mean that I have to manually type out every single category name, every time a new asset gets added? I don’t like typing; I do that all day every day anyways. So why do more of it? We are super lazy. So what do we do? We automate everything!
In this case, we might want to script everything. Now we’ve decided to use scripting. By scripting, I mean that we want to be able to run from a command-line interface. So, we probably also want it to be pretty lightweight. When I think of scripting, I don’t really think of Xcode projects. I don’t want to launch Xcode to just script things. When I think of scripting, I think of this xkcd comic. In this comic, people are having trouble parsing addresses or something. This Tarzan swoops in saying, ”I know regular expressions!”, taps on the computer, and swoops away. That’s what scripting is, not Xcode projects. Last but not least, we want some tools that help us with scripting. We might want something to help us with argument parsing or something to help with indentations while generating code. I don’t want to write all of that on my own.
Using Swift (4:07)
So, now that we’ve decided to do scripting, I was wondering in the Venmo iOS Slack channel whether to write in Ruby. That’s what we’ve written our past scripts in. I thought about Swift, but wasn’t sure if I actually wanted to go down that path. A two day project in Ruby might turn into a seven day project, because I wasn’t very familiar with Swift and scripting. But, maybe looking into Swift might be kind of fun. Then, someone from the New York office responds, “You should totally write it in Swift”. When I heard that, I was like, challenge accepted. So, we did it! Our script goes through our images directory and loops through all the file names. It writes out to a file, like a
UIImage category based on the file names that the designers meticulously pick.
Why did we choose Swift? With Swift, you can work with familiar APIs. I work with Cocoa all the time, and I can use the exact same things that I’m using for app development in scripting. Another nice thing is that it’s a very light feeling language, so it’s similar to Ruby or Python. But at the same time, you have a very protective compiler that I think of as very caring. You get the benefits of both a light feeling language while having the benefits of a very strict compiler, which is the best of both worlds.
There are some other benefits too. You can reduce language dependencies. I briefly mentioned that we have Ruby scripts that we use for other things, and we also have some Bash scripts. I try to avoid touching those because I’m kind of scared of them. But, you can reduce language dependencies. And, if you want to start writing non-production Swift, try running scripts first. If your company or team is afraid of investing time in Swift, you can start experimenting with Swift by writing some scripts. Eventually, you can go full Swift and start writing apps in it.
Demo: The Script (7:12)
How do we actually get started with scripting? If you have a script called
hello-world.swift, the simplest thing to do is run
xcrun swift hello-world.swift. If you want to get a little bit fancier, you can add execute permissions by running
chmod +x hello-world.swift. Then, add the interpreter directive
#!/usr/bin/env xcrun swift to the top of your script file to tell it how to run the file. Now you can just run
I’ve told you how to do things, but let me show you. Our basic
hello-world.swift prints “Hello, world!” upon the command
./hello-world.swift. That’s too simple, so let’s try something a little bit more complicated. I have another script called
hello.swift that takes the first argument and prints out “hello” and whatever you typed in. If we type
./hello.swift swift, it prints out “Hello, swift!”
I think this is still a little bit simple, so let’s try something a little bit more complicated. In San Francisco I gave a talk on how to build a Naive Bayes classifier in Swift. We trained a machine learning model by feeding it with a bunch of examples that you know are something. For example, you can write a spam classifier by giving it a lot of spam and ham examples. You can then give it something new and ask if it’s ham or spam. Now that I’m using scripts, I can give it thousands of examples and train the model to be a lot better.
So, my script creates a new Naive Bayes classifer and goes through four directories. These directories have training data for spam, training data for ham, testing data for ham, and testing data for spam. When we run this script, it will actually go through those one thousand examples each for ham and spam, and then test whether it can identify one hundred examples each of ham and spam. Once it’s done running, we can see that it has marked all of the ham examples correctly, but only forty-two of the one hundred spam examples as spam. It’s not very good at detecting things as spam. So it tends to err on the side of ham, if it’s not sure. You can find these demo scripts here.
Dependency Management? (11:28)
What about dependencies? Why did I have my entire Naive Bayes classifier code in my script? What if I wanted to pass an option to specify where to look for the ham examples and spam examples, or how many to use as training data versus testing data? How can I add different frameworks or dependencies?
When I think of dependencies, there are a couple of things that come to mind. The first one, and simplest, is just Git submodules. It’s simple, but it does very little. For me, I think it’s a little bit too simple. I want a little bit more magic. Then I started thinking, maybe CocoaPods. That’s what I use for all my iOS apps so why not for this? It supports Swift now, as of the 0.36 release, which is super awesome. It also handles dependency management, but it also relies on an Xcode project. In my case, I prefered to code in Vim, and I didn’t want to rely on Xcode. So maybe Cocoapods wouldn’t work out.
Then, I heard of something called Carthage. It supports Swift and also does dependency management. You also have to manually add frameworks to the project, but that might be a good thing for me. I started playing around with Carthage a little bit, and it told me to just create a
Cartfile and declare dependencies. I added PrettyColors and Alamofire, and then I ran
carthage bootstrap. That builds things and outputs them to a build directory in your current directory. So, in
Carthage/Build/iOS there was an
Alamofire.framework and a
Now that I had those frameworks, what can I do with them? Digging around on the Internet turned up the answer of just adding them into your library’s frameworks. What if I didn’t want all of my projects to know about all of my frameworks? I started looking up the Swift docs, and then saw that one of the options lets you specify a framework search path. Because Carthage spits out the frameworks in a specific directory, I just changed the interpreter directive at the top of my Swift file to specify the framework search path.
So, I have a simple script that prints out a poem. I want that poem to be printed in colorful text, so I import PrettyColors. The terminal prints the first line in red, second line in blue, then black, and then blue. Now we have very colorful text output. That was a very simple example of including frameworks.
Missing Stuff (15:18)
What’s missing? You can do a lot of stuff with this already because you can include all these frameworks that support Carthage. When I first wrote that Venmo image asset generation file, I used Vim. I didn’t have syntax highlighting for Swift, and I also didn’t have auto-completion. I don’t really know how I ended up writing it, but I did. It would be nice to have some sort of auto-completion from something like Vim, and maybe a better way to do dependency management. The way I showed you is something I just figured out by poking around the Internet and putting pieces together. It’s kind of a hack, but I like to think of it as a very creative way to use Carthage.
Helpful Tools! (16:20)
We could also use more tools. I think there are lots of things out there, but most people don’t know that you might be able to use Swift for scripting. There needs to be more stuff that everyone can use together. The good thing is, there is already a bunch of stuff out there that can help us. Later, I realized that Keith Smiley wrote a Vim syntax highlighter that I’m now using, and is very helpful.
There is also PrettyColors, Argo for JSON parsing, CLIKit for option pargins, SemverKit for semantic versioning & comparisons, and maybe something called BananaKit. And then, question mark. Maybe you guys can help us write and share tools so that we can write scripts and awesome code together.
Plot Twist (17:08)
This is where my talk was going to end, but there was a slight plot twist this morning. Over dinner last night, I was talking with some of the CocoaPods team members, Kyle and Boris. I told them that in my talk, I would be mentioning Carthage even though I loved CocoaPods. Then, Kyle said that Boris could just write a CocoaPods plugin for that, and he would call it Rome. If you know any history, it’s funny that he would call it that.
I thought it was a funny joke, and I wasn’t going to take it seriously. But at 6:30AM, I woke up and checked Twitter. I saw a mention from Boris with a link to a repo called Rome. I followed the link, and now apparently there is a CocoaPods plugin that spits out frameworks. So now you can use both Carthage and CocoaPods, or whatever you want to do to write scripts for Swift. It has pretty simple usage instructions, and it seems to output everything to the directory Rome.
Q: How much hassle is it to use Foundation or any of the Cocoa frameworks in your Swift scripts?
Ayaka: You can use all of it. The linguistics framework that I use uses
NSLinguisticTagger, which is only on iOS but I can still use it.
Q: Do you have any strategies for versioning Swift with your Swift scripts? Is there any way to specify that your script works on say, Swift 1.1 but not on 1.2?
Ayaka: That is a really good question, but I don’t know what the answer is to that. I’m happy to throw around ideas.
Audience: Maybe you can shell out to the command line and use
xcode-select to change it for the user.
Netsparker Community Edition is a free Cross-site Scripting and SQL Injection Scanner. It’s a free edition of our web application security scanner for the community so you can start securing your website and web applications now. It is user friendly, fast, smart and as always False-Positive-Free.
It shares many features with Professional edition. It can detect SQL Injection and Cross-site Scripting vulnerabilities better than many other web vulnerability scanners, if not all, and it’s completely FREE.
Role : Java Tech lead
Location: Sacramento, CA
Note: We can do CTC, W2 Contract or Full-Time. We can also do H1 Transfer.
ProKarma delivers integrated technology and business process outsourcing solutions for over 150 global leaders in a wide range of industries and markets. ProKarma is co-headquartered in Portland, Oregon., and Omaha, Nebraska., with sales and delivery centers in the United States, India, Argentina and Peru. ProKarma was selected as a Global Services 100 Provider for 2012 and 2013 and ranked as the fastest-growing IT services company in America by Inc. 500. For more information, visit http://www.prokarma.com
Roles & Responsibilities:
• Experience utilizing Java J2EE platform to design, implement, and deploy scalable fault tolerant applications on JBoss Fuse Platform
• Experience developing a System Architecture Document.
• Experience developing design patterns for clustered high availability applications/reusable & scalable objects/optimized data retrieval and storage/efficient code and non-blocking multi-threaded applications.
• Experience providing Web Service design using Representational State Transfer (REST) and Simple Object Access Protocol (SOAP).
• Experience performing Audit Java Security, Authentication, and Authorization of application.
• Experience auditing Java performance a tuning of existing applications
• Experience providing JBoss Fuse Platform diagnostic and troubleshooting ability.
• Experience identifying existing issues and pro-actively recommend solutions for upcoming issues.
• Experience optimizing Hardware Load Balancing when integrated with JBoss Fuse clustering and load balancing.
• Experience providing subject matter expertise through a detailed knowledge of Java, UNIX/LINUX environment from a system administration and development point of view and may be in form of knowledge transfer or more formal training.
• Experience optimizing operating system for high performance multi-threaded applications that require high CPU and network usage and shall diagnose and triage issues with such systems.
• Experience in designing topology for Application Server cluster farm
Knowledge and skills needed:
• Good understanding of J2EE technologies
• Ability to balance multiple projects and priorities
• Innovative, independent, self-motivated, and able to work under tight deadlines
• Strong problem solving, troubleshooting, planning, and organizational skills
• Hands-on lead capable of executing strongly both as an individual contributor and a team lead
• Proven sense of high accountability and self-drive to take on and see through challenges
• Knowledge with JBoss Application Server
• Certification with JBoss Fuse
• Unix shell scripting
• B.S. degree in Information Technology or other technical discipline
Dinesh Ram Kali.
Human Resource Associate| National Staffing|
Direct: +1 (402) 905 9212
222 South 15th Street, Suite 505N, Omaha, NE 68102
Role: Sr. Software Engineer
Location: Bellevue, WA
We need only W2 candidates. We can also do H1 transfer
Note: Send me your updated resume at email@example.com
We are seeking engineers excited about building and scaling infrastructure to evolve our analytics platform. As an engineer with our advertising team you will work with technologies to process and organize our data in batch and real-time contexts. The ideal candidate is an autonomous engineer with extensive knowledge of the software development process and an understanding of distributed systems. Familiarity or past experience with Hadoop and related technologies is required.
• Build solutions to measure, forecast, and analyze ad revenue & performance to help product and business teams make informed decisions.
• Empower our data scientists and analysts to apply statistical methodologies and ask complex questions across large data sets.
• Develop, test, deploy, and support applications using Scala, Java, and Ruby.
• Play a key role in influencing and planning the direction of our data infrastructure.
• Create MapReduce jobs to transform and aggregate data in a complex data warehouse environment. • Write custom code and leverage existing tools to collect data from multiple disparate systems.
• Implement processes to ensure data availability, accuracy, and integrity.
• Strong Java development skills.
• Hands-on development mentality with a willingness to troubleshoot and solve complex problems.
• Experience and knowledge of technologies within the Hadoop ecosystem, such as Cascading, Hive, Pig, HBase.
• Experience with AWS a plus.
• Understanding of Linux system internals, administration, and scripting.
• Interest in and willingness to work with machine learning, Mahout, R.
Dinesh Ram Kali.
Human Resource Associate| National Staffing|
Direct: +1 (402) 905 9212
222 South 15th Street, Suite 505N, Omaha, NE 68102
Role: Senior Java QM Engineer
Location: Beaverton, OR
Note: Interested candidates can send me your updated resume at firstname.lastname@example.org or reach me at (402) 905 9212. Please share or like this post.
What You Need for this Position
• Strong communication skills.
• Experience developing automated integration and performance tests.
• Strong (demonstrable) knowledge and hands-on experience with Java.
• Experience with Linux (CentOS, Ubuntu, etc).
• Experience developing in a continuous integration environment using Jenkins or similar CI frameworks.
Preferred if you have the following:
• Experience maintaining gradle builds.
• Experience using NoSQL databases such as Cassandra, Mongo or Couchbase.
• Experience with Netflix OSS technologies.
• Experience integrating with or utilizing production monitoring technologies such as New Relic or Splunk.