Monday 28 March 2022

Working in Artificial Intelligence and Machine Learning at Electronic Arts and Bioware Presentation, March 25, 2022 (University of Alberta)

The actual title of this talk was “Machine Learning at Electronic Arts and Bioware”, but I changed the title of the blog to emphasize the “Working in” component. That’s because it seemed to me that the talk was tailored to an audience (mostly university students and staff) who would be interested in knowing what it was like to work in data science at these companies, what data science techniques were used in their businesses and what qualifications would be expected.

I should note that I am just a statistician/data scientist, well along in my career, so I am not interested in a job in this field, just interested in the subject. But, for any younger people interested in data science and computer gaming, this sounds like a great line of work. I should also note that there are positions for non-IT people in these fields - for example, a friend of my son is a writer for Bioware (i.e. game scripts, plots and things of that nature). She seems to like it.

Introduction

As many probably know, Electronic Arts (EA) and Bioware are computer game companies (actually Bioware is now wholly owned by EA).
Bioware started off as a local company in Edmonton, Canada, which is also the home of the University of Alberta (the U of A has been prominent in academic computer game research for many decades). There are a lot of cross-linkages among these two entities. Bioware’s most successful game is probably “Mass Effect”.
Electronic Arts is based in Florida and has produced a plethora of computer games, and is particularly strong in sports-themed games (NFL football, etc).
The talk, done by Bioware’s Alex Lucus and EA’s Bill Gordon, was centered on the use of machine-learning (ML) artificial intelligence (AI) algorithms within their companies, for game development and other purposes.
It also outlined the connections and partnerships that each company had with government research agencies (e.g. National Research Council) and various academic institutions.
It was stressed that there are many opportunities in the gaming world for computer professionals in all stages of their careers, though this talk was especially targeted at students and early career individuals.
Given the nature of the talk, it focused on opportunities for those with data science backgrounds. Nevertheless, they stressed that their are many other openings for computer professionals in their companies.

Machine Learning Strategies at Electronic Arts (Bill Gordon)

The speaker has a B.S.E.E. (electrical engineering) background and about 8 years experience with Electronic Arts, based in Orlando Florida.
He has been involved in many areas that make use of AI and ML at EA, particularly AI for games development and verification.
He started out with game development, but is now with the AI support team, which supports all of the company’s teams.
He stressed that AI covers a lot of ground at Electronic Arts, including:
- Game development (e.g. adversarial AI, where the game “plays itself” to learn new strategies).
- Defect detection within games (looking for errors or inconsistencies within the game, such as a football player with no helmet).
- Quality Assurance (it has to be bug-free and responsive).
- Non-game areas, such as marketing and customer support.
Game software has a number of requirements, for which AI and ML can be useful:
- Strict performance standards (e.g. “real-time” playing and realistic appearances).
- Stability (no crashing or hanging).
- Timing and synchronization (needing to work well in stand-alone, internet server environment, and mobile apps.
There is a mix of traditional AI and newer machine-learning based methods being used, which complement each other. So, a developer should understand both.
- Traditional AI is basically coding in rules and logic to a game, rather like a traditional expert systems approach.
- Machine Learning AI can include such aspects as having a non-human opponent in the game, playing “other” characters within the game, or giving hints or instructions to the human players.
Electronic Arts has tended to take a “small steps” approach to introducing ML to its games.
The focus has been “micro” rather than “macro”. For example, rather than attempting to revamp the entire football game via AI, the focus could be on using AI to train a particular game player, within the game. For example, AI could be used to train running backs on “finding a hole”. It could be counter-productive to try to do too much all at once.

Machine Learning Algorithms Being Used for Game Development and Testing

Before getting into the details of ML usage at EA, the speaker emphasized some general matters about coding and AI:
- The company wants practical applications for advancing the use of AI for matters such as game playing and testing.
- This means that developers should have a deep understanding of the algorithms and the code used to implement them.
Reinforcement Learning
- Reinforcement learning is based on an “agent” (i.e. a computer generated game player) playing many iterations of a game or part of a game, to improve its game.
- Perhaps the classic example of this is AlphaGo, which played itself millions of times, and discovered strategies to exploit for success, which human players had never thought of (and thus beat Lee Sedol, the leading human Go player).
- The agent has a reward function, which is calculated during each iteration of the game or partial game. The algorithm will have the agent try various strategies, to determine which ones offer the most reward (depends on the exact way that the reward function is operationalized for any given game). Those strategies will then be preferred.
- An agent may do odd “outlier” things that a regular testing regime might miss. For example, a golfing agent might hit the ball backwards, which is a scenario traditional testing might not anticipate, but real human players sometimes do.
- So, reinforcement learning can help with both “seeking exploits” for game strategies and testing game stability.
Imitation Learning
- This involves having the algorithm learn techniques that have worked for skilled players in the past, and imitating them.
- Again, using the AlphaGo example, the computer was fed many games between the best humans, as a basis for learning excellent playing strategies. Those could then be augmented by reinforcement learning.
- For example, in football, real players generate an enormous amount of data (e.g. wearing RFD tags) that EA sports can feed to an ML algorithm, so that if can learn successful strategies.
- A computer-generated quarterback can then imitate real human players (e.g. Tom Brady), and learn how a computer spots an open receiver.
- Imitating the real-life actions and behavior of players can also make the game look more realistic, with smoother flow to motions and more genuine-appearing background play.
- This ML learning can sometimes outperform traditional physics-based AI simulations. For example, imitations, based on thousands of examples of a throw to a receiver on the sidelines, might yield a more realistic appearing arc to the pass than physics-based calculations. The program might also execute faster, which is important in real-time game simulations.
Image Classification/Object Detection
- This involves some overlap between development and testing.
- Image-classification and Object Detection ML can be useful in detecting anomalies in the game (e.g. a football player who doesn’t have a helmet on, or perhaps doesn’t even have a head).
- This is rather like an automated “continuity girl” in movie production (someone whose job it is to ensure consistency between scenes).
- So, it can be used to validate assets or catch bugs.
Natural Language Processing, Numerical Analysis, ML Research in General
- Most games have audio and conversation, so NLP can be used for such matters as having agents make relevant natural-sounding conversation during the game.

Working at Electronic Arts and Bioware

Perhaps one of the main messages of the talk was that you can get a job as a game developer at companies like EA and Bioware.
They have needs for both traditional programmers and data science professionals.
Main skills needed:
- Programming (obviously), especially C++.
- That said, programming languages change constantly, so flexibility is vital. Python is also gaining in popularity. Nobody knows what the future may hold.
- AI and ML skills and knowledge are also obviously important, but that should be in tandem with excellent coding skills.
- Gaming is a real-time programming task, so it is important that code runs fast and reliably and is easy to maintain.
- A developer needs to be a quick learner and to have important character attributes such as integrity, adaptability and the ability to get along in diverse teams.
Some areas in Game Development teams:
- game play
- graphics
- online game developments
- systems/analysis.
Other areas of IT and Data Science employment with Gaming Companies:
- finance and marketing.
- security.
- tools.
- translation.
- QA.
- data storage.
- database development, especially server and “big data”.

Questions (from the audience, which was heavily skewed to university students) and Answers

What AI Research is going on at Electronic Arts?
- There is plenty of research going on (e.g. papers/speakers).
- Some is of the general open-source variety but some is very EA-specific.
- So, it is both public and internal.
- An example: a PhD student did research on facial expressionless and gestures, which the company used to help sync these up, for characters in a dice game).
- There is also research into animation/rendering and cheater detection.
- The company has huge quantities of data (video, text, etc.) from its games, there there are many possibilities for research.
How about at Bioware?
- Bioware has many research partnerships with government agencies such as NSERC (Natural Sciences and Engineering Research Council of Canada). Many of the problems and research opportunities that interest Bioware also interest research students and academics. The NSERC CREATE grant has been very useful in this regard.
- Check with Alex.Lucas@bioware.com if these opportunities interest you.
What are some of the AI ML techniques that you use?
- Bill Gordon was not up-to-date on what was being used for strategy games, though he was aware of some experimentation with reinforcement learning.
- A lot of sports statistics, and RFD tag data (actual players with radio-frequency-tags embedded in uniforms) are used to generate data for game-related analysis, animation, etc..
Are you concerned about DeepFakes? Is this a problem for EA?
- Yes, there is some concern. For example, users can augment aspects of games (e.g. racing car slogans). This can be done in inappropriate ways, that could include DeepFakes.
- So, the company has to adapt and find new ways to identify harmful augmentations (e.g. NSFW, biased, defamatory content).
What about jobs in gaming?
- There definitely are jobs with Game companies like EA and Bioware.
- Not all jobs are directly related to games, though. Gaming companies have other important teams that need data science input (e.g. marketing).
- You need the desire to solve difficult problems, in all sorts of domains.
- Software of this type has to be quick performing.
- It does involve “hard work”.
- You need to be willing to learn and keep on learning.

Some related blogs of mine:

1) The Movie Colossus, the Book Superintelligence and Artificial Intelligence:

https://dodecahedronbooks.blogspot.com/2018/12/the-movie-colossus-book.html

The movie (about 1970) is about an AI taking control of the world, the book(about 201) is about the prospect of the same. Great movie, way ahead of its time.

2) U of A Lecture – Demystifying Artificial Intelligence- Part 1, History of AI: http://dodecahedronbooks.blogspot.ca/2018/03/u-of-lecture-demystifying-artificial.html

3) U of A Lecture – Demystifying Artificial Intelligence- Part 2, Varieties of AI: http://dodecahedronbooks.blogspot.ca/2018/03/u-of-lecture-demystifying-artificial_3.html

From other talks, also given at the University of Alberta, on the general subject of AI and games.

4) Computer vs Human in Go: http://dodecahedronbooks.blogspot.ca/2016/03/go-match-alphago-versus-lee-sedol.html

A 5) A Smart Move – AI and Strategy Games. https://www.blogger.com/blog/post/edit/8998037621231280699/5460797801205874261

Now I will plug a book or two:

If you want to get away from computer games and see the real world, you might want to consider a nice road trip, exploring the contrasts between Canada and the U.S.. If so, then “On the Road with Bronco Billy” is definitely your book.

Sit back and go on a ten day trucking trip in a big rig, through western North America, from Alberta to Texas, and back again. Explore the countryside, learn some trucking lingo, and observe the shifting cultural norms across this great continent. There's even some hockey playoff talk (Oilers-Denver and Oilers-Dallas), for those nostalgic for Canadian playoff representation.

It’s on Amazon (ebook), for a mere 99 cents (U.S.).

Amazon U.S.: http://www.amazon.com/gp/product/B00X2IRHSK

Amazon U.K.: http://www.amazon.co.uk/gp/product/B00X2IRHSK

Amazon Germany: http://www.amazon.de/gp/product/B00X2IRHSK

Amazon Canada: http://www.amazon.ca/gp/product/B00X2IRHSK

Or, if you want to continue contemplating strategic games, you could try “A Dark Horse”, which concerns the troubling run of good luck that a (fictional) horse player experiences, and his extraordinary opponent.

Also on Amazon (ebook), for 99 cents (U.S.).

Amazon U.S.: https://www.amazon.com/dp/B01M9BS3Y5

Amazon U.K.: https://www.amazon.co.uk/dp/B01M9BS3Y5

Amazon Germany: https://www.amazon.de/dp/B01M9BS3Y5

Amazon Canada: https://www.amazon.ca/dp/B01M9BS3Y5

Monday 14 March 2022

Pi Day – Euler’s Infinite Expansion of Pi

Most people know that pi is a number that is infinite and non-repeating i.e. it never ends and there is no repeating pattern to it. Numbers like these are known as irrational or transcendental, depending on the type of equation that can be solved by them. Pi is actually transcendental, though it took a long time for that to be proved mathematically.

Though pi is known by most people as the ratio of a circle’s radius (or diameter) to its circumference, it actually turns up all over the place in mathematics. One of these places is as the solution to infinite series, of which many have been discovered.

A famous result was discovered by the mathematician Leonard Euler, in 1734. It says:

1 + 1/2²+ 1/3²+ 1/4²+ 1/5²+ … = pi²/6

In other words, if you take all of the integers and square their reciprocals, then add them up, the answer will be exactly equal to pi squared divided by 6.

A related result can be derived for the even integers:

1 + 1/2²+ 1/4²+ 1/6²+ 1/8²+ … = pi²/24

For the odd integers:

1 + 1/3²+ 1/5²+ 1/7²+ 1/9²+ … = pi²/8

All of these results can be proved analytically, though you have to know some calculus, as the proofs involve integrals. Rather than do this, I set up some R code to explore the expansions, and see how close they come to the value of pi.

There are a few obvious cautions, both involving infinity:

We can’t actually add up an infinite number of terms, so we can never get the exact sum for the various series.
We don’t actually know the exact value of pi (and can’t ever know it), so we can’t actually do the comparisons.
Computers will run into problems with floating point operations as the numbers become very large or very small.

Nonetheless, even a modest laptop computer can provide some interesting results, as shown below:

The top table shows show:

the number of terms in each trial of the expansion (from 10 to one hundred million),
the actual value of pi (as approximated by the R program),
the value of pi using all integers from1to the upper limit of terms,
the value of pi using the first “limit” number of even integers,
he value of pi using the first “limit” number of oddintegers

The second table is similar, but it shows the difference between R’s value of pi and the value computed by the expansions, expressed in parts per billion.

As you can see, the value that most of us routinely use (3.14) is well approximated after about 1000 terms of these series. For those of us that like to be a little more precise and remember the value to five decimal places (3.14159), on the order of 100,000 terms is needed.

Eventually, by 100,000,000 terms in the expansions, the difference between R’s “true” value of pi and our expansion approximation is down to less than 10 parts per billion. Thinking of that in more everyday terms:

If you drew a circle 1,000 km in diameter, 1 part in a billion is about 1 mm.
In a circle of 360 degrees, 1 part in a billion is about one-thousandth of an arc-second. The resolution of the Hubble space telescope is about one-twentieth of an arc-second.
If you had a pumpkin pie that weighed 1 kilogram, 1 part in a billion would be about 1 microgram. A particle of the pie that was one-tenth of a millimeter in diameter would weight about that much.

A couple of interesting results that I wasn’t expecting, are that the sum of odd integers converges faster than the sum of even integers, and even converges faster than the sum of all integers. Also, the “all integers” and the even integers converge at the same rate. So, by 100 million terms, the odd integers have converged to within 3 parts per billion, while the other two are still at about 9 parts per million.

Those results definitely seem counter-intuitive, but I can’t see anything wrong with my R code (given below). It is something to think about, and shows how math sometimes gives results that seem strange, at least at first sight.

Sources:

Inside Interesting Integrals, Paul Nahin, Springer (p 154-55)

Here is some R code that you can run and experiment with, to see for yourself:

# -----------------------------------------------------------

# R program to estimate pi from Euler's formula,

# that uses an infinite series.

# ---------------------------------------------------------

ls()

options(digits = 11)

# the dataframe is intialized with the first, short expansion.

limit <- 10

# All integers

pi_est_all <- seq(1, limit, 1)

pi_est_all$a <- 1/(pi_est_all*pi_est_all)

pi_all <- sqrt(6*sum(pi_est_all$a))

pi_all

# Even integers

pi_est_even <- seq(2, limit*2, 2)

#pi_est_even

pi_est_even$a <- 1/(pi_est_even*pi_est_even)

pi_even <- sqrt(24*sum(pi_est_even$a))

pi_even

# Odd integers

pi_est_odd <- seq(1, limit*2+1, 2)

#pi_est_odd

pi_est_odd$a <- 1/(pi_est_odd*pi_est_odd)

pi_odd <- sqrt(8*sum(pi_est_odd$a))

pi_odd

pi_init <- cbind(limit, pi, pi_all, pi_even, pi_odd)

pi_init

pi_est2 <- pi_init

pi_est2

# Loops through, with other limits for the expansion. This increases the limit

# by a factor of 10, for each trip through the loop. The limits and increments

# can be changed. Computer will run out of memory at some point.

limit <- 100

while(limit < 1000000000)

{

# All integers

pi_est_all <- seq(1, limit, 1)

pi_est_all$a <- 1/(pi_est_all*pi_est_all)

pi_all <- sqrt(6*sum(pi_est_all$a))

pi_all

rm(pi_est_all)

# Even integers

pi_est_even <- seq(2, 2*limit, 2)

#pi_est_even

pi_est_even$a <- 1/(pi_est_even*pi_est_even)

pi_even <- sqrt(24*sum(pi_est_even$a))

#pi_even

rm(pi_est_even)

# Odd integers

pi_est_odd <- seq(1, 2*limit+1, 2)

#pi_est_odd

pi_est_odd$a <- 1/(pi_est_odd*pi_est_odd)

pi_odd <- sqrt(8*sum(pi_est_odd$a))

#pi_odd

rm(pi_est_odd)

pi_est <- cbind(limit, pi, pi_all, pi_even, pi_odd)

pi_est2<-rbind(pi_est2, pi_est)

limit <- limit * 10

}

# Convert to a data frame and print results.

pi_est_df <- as.data.frame(pi_est2)

pi_est_df

ls()

And, here are some earlier blogs about pi, where it shows up in other areas of math, namely geometry, statistics and number theory.

https://dodecahedronbooks.blogspot.com/2020/03/pi-day-2020-pi-and-normal-distribution.html

https://dodecahedronbooks.blogspot.com/2019/03/pi-day-2019-shooting-arrows-at-target.html

https://dodecahedronbooks.blogspot.com/2018/03/pi-day-2018-prime-numbers-and-pi.html

https://dodecahedronbooks.blogspot.com/2017/03/pi-day-floor-pie-and-floor-pi.html

https://dodecahedronbooks.blogspot.com/2016/03/pi-day-31416-some-eerie-pi-coincidences.html

https://dodecahedronbooks.blogspot.com/2015/03/pi-day-31415-pi-and-science-fiction.html

So, now that you have done some math, you should read a science fiction book, or even better, a whole series. Book 1 of the Witches’ Stones series even includes a reference to pi.:

Kati of Terra

How about trying Kati of Terra, the 3-novel story of a feisty young Earth woman, making her way in that big, bad, beautiful universe out there.

http://www.amazon.com/gp/product/B00811WVXO

http://www.amazon.co.uk/gp/product/B00811WVXO

The Witches’ Stones

Or, you might prefer, the trilogy of the Witches’ Stones (they’re psychic aliens, not actual witches), which follows the interactions of a future Earth confederation, an opposing galactic power, and the Witches of Kordea. It features Sarah Mackenzie, another feisty young Earth woman (they’re the most interesting type – the novelist who wrote the books is pretty feisty, too).

https://www.amazon.com/dp/B008PNIRP4

https://www.amazon.co.uk/dp/B008PNIRP4

The Magnetic Anomaly: A Science Fiction Story

“A geophysical crew went into the Canadian north. There were some regrettable accidents among a few ex-military who had become geophysical contractors after their service in the forces. A young man and young woman went temporarily mad from the stress of seeing that. They imagined things, terrible things. But both are known to have vivid imaginations; we have childhood records to verify that. It was all very sad. That’s the official story.”

https://www.amazon.com/dp/B0176H22B4

https://www.amazon.co.uk/dp/B0176H22B4