Presentation Feedback

Question is slightly unclear. The word better should not be used as it is not objective. Also the possibility that a more complex usage of G.A. may increase overall complexity quite a considerable amount.

Presentation

How can Genetic Algorithms best be used to create non-playable characters which are capable of adapting to better compete against different types of player in a real time game environment and what are the issues associated with their use?

Matthew Lock

Technical Supervisor: Dave Ellison

Create A.I. entities that are capable of adapting
Entities must remain flexible
Entities need to improve while remaining realistic
I.E. should not be “overpowered”
Performance hindrance should be minimal

Entity can learn at runtime
Makes play more player specific
Makes entities more human like

Multiplayer game – Based on Bomberman
Implement Genetic Algorithm for A.I. players
Use kills and survival as measure of success
Compare with a Basic “A.I.”

Fairly advanced entities.

Worksheet 4 Feedback

(Also old)
An interesting topic.
I think you understand the issues very well with this topic.
The question is a little long. I would truncate the last part as it should be implied by answering the first part of the question, so it would become:
How Can Genetic Algorithms best be used to create non-playable characters which are capable of adapting to better compete against different types of player in a real time game environment?
I think you might be taking on a lot of work here. Not only do you have to create a game, but also several different genetic algorithms. I would maybe just choose to of the most common genetic algorithms and do a comparison between them.
It is very good that you have recognised the judging criteria of the AI, though this might not be enough. You might want to have an aggressiveness factor included too. The reason is an AI could survive for a very long time by hiding from all opponents. This would make it a good strategy though a boring one which could infuriate a player.
Carry on the good work, I hope it goes well.

Worksheet 4

Worksheet 4 (old adding now as had forgotton to upload it)
Introduction
The topic of the project is the usage of genetic algorithms. The aim of the project is to use genetic algorithms to create A.I. entities capable of altering behaviour in a real time game environment to better compete against players or other A.I. entities, while remaining relatively flexible. Also numerous different variations will be implemented to allow further insight into what works best in a given situation as well as into the issues involved with the methods.
Issues
The issues to be addressed are how best genetic algorithms can be used to create A.I. entities that can adjust to perform at a better standard in a real time game environment. It is vital that such entities remain flexible and are realistic in there playing ability, i.e. they should not have super-human ability at the game. Also because of the real time aspect it is important to consider any impact on performance, alongside any other issues that may be involved in the implementation.
Research question
How Can Genetic Algorithms best be used to create non-playable characters which are capable of adapting to better compete against different types of player in a real time game environment and what are the issues associated with their use?
Addressing the Question
I will write a multiplayer game, likely to be based on Bomberman, that will contain enough decision making and strategy for the computer controlled players to be able to learn, through playing, which techniques and strategies are the best way to play the game. I will then use several varying techniques to control this A.I. learning and judge them on success, based primarily on number of kills but also on time survived, as well as on flexibility and effect on performance. This will allow me to figure out which methods work best as well as evaluate any issues with the techniques.
Progress
So far I have researched around the area of genetic algorithms as a means of adaptive A.I. to help redefine exactly what I’m doing. This has enabled me to better aim my work and to understand what I am doing and what I need to do from now on.

Feedback

A.I. is ever the hot topic in the industry. Adaptive A.I. is even hotter and seems to be the next big thing and it will be. I think though, personally, Adaptive A.I. will be a progression and it’ll start off small in A.I. and then keep growing and growing. Firstly it’s good that you defined what adaptive A.I. is not: to create such a human like A.I. it’s a lot harder to beat, which is completely not the point as some people expect. So this shows you know what you’re researching at least. The only thing I think I should say is that to keep it pinned down, hard. It’s such a broad topic in the industry right now. Which is good as you’ll be able to find a lot of materials for it. It’s also bad though that you might get lost in it.

Bomberman is quite a good choice to base you’re A.I. code on. A lot of people have played it and know how it plays and a new method of A.I. should be noticeable. Are you going to do testing? Will it involve a spread of people? This will allow you to test your A.I. code in the best possible way and will give you feedback on what people think. Bomberman won’t be easy though as the game progresses the amount of choices available to the player increase tenfold. Yet this won’t be impossible to overcome =using some slecetion techniques.
This, as always, is a good topic to researchc and hope that it will bear fruit for you and I look forward to playing some bomberman.

Worksheet 3

Introduction
The aim of the project is to use genetic algorithms to create A.I. entities capable of altering behaviour in a real time game environment to better compete against players or other A.I. entities, while remaining relatively flexible. Also numerous different variations will be implemented to allow further insight into what works best in a given situation as well as into the issues involved with the methods.

Motivation
Artificial intelligence is becoming more and more important as games attempt to create further realism. By using adaptive A.I. it is possible to have an entity “learn something at runtime from a source of information (such as the person who bought the game) that is unavailable at development time.” (Funge, 2004) Because of this it is possible to create a game entity that can change dependent upon the person playing the game, to better play against the player. It is however important to note that the aim of artificial intelligence is not to create unbeatable opponents but to create “algorithms that make game characters appear human or animal-like.” (Millington, 2006) The motivation behind this project is to explore learning A.I. in an attempt to better learn how best these two elements of A.I. can be brought together to create an A.I. that will learn and react while remaining flexible and not becoming overpowered.

Research Question
How Can Genetic Algorithms best be used to create non-playable characters which are capable of adapting to better compete against different types of player in a real time game environment and what are the issues associated with their use?

Addressing the Question
I will write a multiplayer game, likely to be based on Bomberman, that will contain enough decision making and strategy for the computer controlled players to be able to learn, through playing, which techniques and strategies are the best way to play the game. I will then use several varying techniques to control this A.I. learning and judge them on success as well as flexibility and effect on performance. This will allow me to figure out which methods work best as well as evaluate any issues with the techniques.

Resource Requirements
I only really need a reasonable level PC for the project.

Bibliography/References
John David Funge. (2004) Artificial Intelligence for Computer Games: An Introduction. [Online] Available at: http://books.google.co.uk/books?hl=en&lr=&id=jy_q4URyrowC&oi=fnd&pg=PP13&dq=artificial+intelligence+game&ots=1jzBSq1QLY&sig=8SWQZJMO7W4mS0Io5AkvBwqoy5E#PPP1,M1. [Last Accessed 04 November 2008]
Ian Millington. (2006) Artificial Intelligence for Games. [Online] Available at: http://books.google.co.uk/books?id=zCiv-lMad-AC&dq=artificial+intelligence+game&lr=&source=gbs_summary_s&cad=0. [Last Accessed 04 November 2008]

Feedback

I recieved the followng feedback:
2a. I think this is a fascinating area of research (although disturbingly Hollwood-esque with A.I. Learning and passing on knowledge). The above paragraph does give a general idea of what you want to achieve, but could benefit from some examples after “possible options” and “inputs” to give early clarification of what you want to achieve. Although, I think such details are explained later on (especially 2c).

2b. You might also want to research the best way to test your implementations , such as a family of characters trying to stop a player or two families competing against each other to achieve dominance.

2c. Since it has such a wide variety of tactics as well as allowing a lot of modifications in your A.I.'s individual personalities, Bomberman would make a good test for you A.I.'s improvements. I made a similar game for my PS2 project and it can get quite complex, especially if you add powerups to be randomly placed around the level. I'd recommend starting to plan and program it now or finding an open-source version that you can use.
Some of the information in this paragraph may be better placed in 2a as - especially the lines about not creating unbeatable A.I.. “Creating balanced but adaptive A.I.” could be considered a focus of the project.

3. You may need to clarify exactly how you are going to define you A.I. as “better”. In the case of Bomberman, would it be the amount of enemies destroyed or how long it can survive? Or would you want to aim for as high a kill/death ratio as possible? If you play against the player, you will have to note that the player's skill will be increasing with each subsequent game. You might also have to arrange testing sessions with volunteers to get as many results as possible.

I think it is an interesting honours subject, with a wide scope for testing. It also has a lot of options for critical analysis through observation of any bizarre behaviour that you A.I. develops and then potentially passes down through the generations. I'd just make sure you don't spend too much time developing Bomberman so that you leave enough time to develop your A.I. to the desired level of complexity.

Worksheet 2

For the second worksheet I wrote the following:
1. How Can Genetic Algorithms best be used to create non-playable characters which are capable of adapting to better compete against different types of player in a real time game environment and what are the issues associated with their use?

2a. Using Genetic Algorithms to create enemy A.I. capable of altering behaviour in order to better compete against a player or other A.I. in a real time game environment. This will involve a scenario where there are numerous possible options for what the character should do, as well as numerous "inputs" which will help it decide. Theses inputs will be used to choose from the options based on a set of "genes" that the character has. The most successful characters will then pass their "genes" onto later characters which should mean that new characters evolve in the direction of greatest success.

2b. The usage of Genetic Algorithms and the various ways in which they can be implemented. This includes things like chromosomes, genes, crossing over of chromosomes and mutations. This will also include how the characters will make decisions based on their "genes".

2c. I will need to create a multiplayer game with a reasonable amount of decision making. There will need to be numerous stimuli for the characters to take note of and numerous options for them to choose based on these stimuli. Then several different variants of a Genetic Algorithm will be implemented to allow the non-playable characters to evolve, and these will be compared, along with a test case containing no evolution, to see which provides the best evolution in a way that positively impacts gameplay, while not having too large an effect on performance. The A.I. will be evolved over a number of generations and the results will then be able to be compared. The optimal results for the experiment will be to create A.I. entities which adapt to counter a players tactics while still being flexible enough to perform well if a player changes tactics. However it is important to note that the aim is not to create an unbeatable A.I. and as a result modification of the standard genetic algorithms, which seek the optimum performance as standard, may be necessary. To provide suitable range for the A.I. I am thinking about testing the Algorithms in a game based on Bomberman. This would allow decisions between being offensive or defensive, decisions about going for powerups, and which powerups to go for as well as other strategic decisions. This would allow for a fully developed A.I. with plenty of scope for adaption while still being a relatively simple game and hence allowing me to concentrate mostly on the A.I. itself.

3. Once the various methods of implementing Genetic Algorithms have been researched and one has been implemented within the demo application it will be relatively simple to adjust the way in which the algorithm works to provide numerous variants. These can then all be put through stages of testing to see which provides the greatest gain at the lowest cost. Throughout the experimentation it is likely that any issues with the Algorithms will become apparent and so this too can be documented.

It is worth noting that after my meeting with Dr King and further research the aim of my project has shifted a bit from what has been mentioned in previous posts.

Meeting and further thoughts

Today I met with Dr King to discuss my idea for my project.
He has suggested that I may want to look into other areas of A.I. as well as Neural Networks, and specifically mentioned the usage of Genetic Algorithms. It was suggested that I read around a number of areas of A.I., and see what other people have done in the area. This will help me decide exactly what I want to do and how I intend to go about it.
Basically my plan now is to research more fully into A.I. techniques, specifically ANN and GA and try to further narrow down my aim for the project. I still intend, as stated before, to create some sort of NPC that will interact with/react to the character, and I am mainly now trying to decide exactly how this will work and what will actually be done for the project.
This includes the practical element as well as it will be important to decide how I am going to determine whether the practical is "successful" and as a result I will need to think about ways to do this. If I use the GA, then I will be able to compare the Adaptive A.I. that the GA's create to the original A.I. without the adaption. However it will be important to do this in such a way as it is visibly different, and would be best if there was an obvious method of measurement, so as to fully demonstrate the differences. Another thing to consider with A.I. is that there are often numerous ways to go about things, even including a drawing together of parts of different techniques (such as using one A.I. method to teach the entity offline and another to adapt it online), and as a result it may be possible to make comparisons to similar studies where similar techniques were used but in a different way. However in that case it would be important to ensure that the two methods were different and that enough information was provided to make a proper comparison.
On the practical front I also need to consider what I am going to use as a basis for my application.

Feedback

I recieved the following feedback:

This is a great topic to look into for an honours project because if the neural network works then they can find themselves more useful instead of using classical A.I. techniques. I'm pretty sure there is a lot of stuff on the web you can use for this. So would it be fair to say for when you research into could you talk about why neural networks can become a powerful tool and talk about the parts that out date classical AI techniques? Also could you talk about the practical uses of NN in the industry and vive versa. This link should give you plenty to think about for your research:

http://aigamedev.com/discussion/useful-neural-networks

I think there is plenty on this topic to do for a honours project and i'm looking forward to see what your final "output" is. :)

here are just a few more links:

www.learnartificialneuralnetworks.com/

http://aigamedev.com

http://search.techrepublic.com.com/search/neural+network+algorithm.html

Initial Thoughts

My initial Ideas are:
The topic I am considering is Artifical Intelligence, specifically Neural.
The main issue I am considering is involving Neural A.I. and how it can be used to make more realistic decisions within games.
My initial ideas in relation to the practical aspects involve using neural A.I. for some sort of game entity that would have to interact with/react to the user. This would have to be something that required a number of different types of input into the system as well as a number of possible outcomes for each situation. It would probably also involve some level of tweaking to alter the strength of certain inputs and hence which outputs are chosen. One thing I have considered is using neural A.I. for an NPC that would react to certain conditions within the game world, similar to say Oblivion. There would be numerous things that the entity would take into account to decide what to do, and different "types of person" would be able to be created by adjusting the strengths of the inputs.