My thoughts on the 13.2.1 Match Engine

[TheSimeon]

So I've only been playing since '08, but the moment I loaded the game up, I was hooked. And, since '09, the match engine has seen refinements, improvements, but no major overhauls. And we got good at it. Like most people I've had an undefeated season and won a triple or quad, because the match engine as constituted had gone as far as it could go, and we had all learned how to exploit it.

And as the min/max-er type gamer, I had tested the crap out of the 09-12 ME to answer some questions I had. A few of the things i figured out that most will know, just for giggles (and you'll just have to trust that my methodology is sound, but these results come from many, many season both simmed and played out, and compared with controls, baselines, etc. I can go into it if anyone needs me to):

1. Always use the match prep attacking movement. it's worth almost a .8 goal differential per game, almost twice as much as any other match prep. Home or away, doesn't matter, it just effects too many stats.

2. If you were take a team and max out any one stat, the most bang for the buck is obviously pace, but one of the most under-appreciated is aggression, which when maxed out is hugely influential in win totals.

3. Increasing a strikers "off the ball" does almost nothing for his match ratings over a couple of seasons, but increasing a midfielders will really make him play better.

4. Shoots with power is SLIGHTLY better than places shots, not because your striker will score more, but because shots that rebound off the keeper will result in just a FEW more goals.

5. If you take any position, and improve the overall rating of that player by 20 points, the most "important" or influential position, is striker. But the second most important is fullback.

There are a LOT of things I tested, but I mostly mention them to point out that I'm not the "OMG my taktiz aint werkin!" type.

And then fm 13 came out. At first I was appalled at the bizarre defending, tendency of players to run away from ball in the air, or unwillingness of players to turn around with the ball, but I also saw (and posted about) the potential of the ME. It LOOKED LIKE FOOTBALL. I attribute this mostly to 2 changes:

1. Collision checking was changed, which made pacy strikers far less effective, and made other types of play besides scoring on the break viable (I know, I know, there were always viable alternatives, but they were all sub-optimal, and we all proved it over and over)

2. First touch was dramatically altered, which drastically cut down on the 'video game sticky foot' phenomenon.

I was encouraged. Just iron out the kinks, and we had a real leap forward in Football Manager.

Then came 13.2.

Honestly, I'm not sure what went wrong. I think that there must have been panic in the SI offices at release, when they couldn't iron out the defending problems. They fixed the hilarious dribbling errors, the ball tracking, player vision....

But they couldn't fix the defending, and so instead they seem to have changed offensive decision making, and right then the game went in the shitter.

What do I mean? Well, lets take emotion out of it, and look at some empirical testing. I noticed that offensive players seemed less imaginative, so I did a few tests.

Q: Does increasing a players creativity or flair attributes improve his performance, or my match results?

A: Not any more. When simming a season, then taking the MC, AMC, AML, AMR, and ST, and increasing all of their flair or creativity ratings to 20 and simming the season again, (and doing this with different sides, tactics, and even playing the games out) the results are clear: Creativity and flair's co-efficient of correlation to winning or match rating is non meaningful, and in some tests, even 'negative'. To be fair, even in fm 12, those stats were less important than most people thought, but now they LITERALLY HAVE NO MEASURABLE POSITIVE EFFECT.

Q: Do players take fewer through balls, as seems obvious?
A: I can't test this directly, as I don't have a game running 13.1 anymore, but i can say this: in FM 09-12 the BEST PPM, in term of match rating and win probabilty, was, by a large margin "Tries Killer ball often". Nothing else was close. In ME 13.2, Tries Killer balls often has NO MEASURABLE EFFECT. Through balls simply don't happen enough, or aren't smart enough, to mean anything any more.

Among the other things I have noticed: Playing a game with the "work ball into box" shout on does not result in measurably fewer long shots that keeping the shout off. Also, taking any tactic from FM and reducing the "creative freedom" slider by 5 clicks results in a more effective tactic more than 90% of the time. I haven't had the time or patience to really get into this, but I think most people who play the game can see just how painfully obvious player decision making has gotten.

Why these results? I'm honestly not sure. Maybe I'm just doing something wrong. Maybe I'm rushing to judgement But I'm not making these observations up.

But my theory is that they couldn't fix the defensive AI. And so rather than watching a stupid AI defense get torn apart every single time someone played a one-two, they made attacking AI dull, predictable, backward.

The result: it looks LESS LIKE FOOTBALL, more like ARTIFICIAL INTELLIGENCE. SI is a great company, and they will figure it out. But lets all join together and encourage them to embrace the chaos for the moment, because as consitituted, the game isn't broken...

It's just dull.

[GodCubed]

You honestly believe that the FM09 match engine is superior to FM13's?

Tell you what, try posting this on the SI forums. I'd like to see what the people who actually made the game have to say to this, since I GUARANTEE you their testing is far, FAR more thorough than yours.

And yes, I would quite like it if you could go through exactly how you tested all of this. Presumably you had large sample sizes, thorough testing, no bias, eliminated as much chance as humanly possible, et cetera?

[TheSimeon]

You honestly believe that the FM09 match engine is superior to FM13's?

Tell you what, try posting this on the SI forums. I'd like to see what the people who actually made the game have to say to this, since I GUARANTEE you their testing is far, FAR more thorough than yours.

And yes, I would quite like it if you could go through exactly how you tested all of this. Presumably you had large sample sizes, thorough testing, no bias, eliminated as much chance as humanly possible, et cetera?

No I don't remotely think that the FM 09 engine is better than this one, and I feel pretty sure I didn't write that. What I do think was that the 13.1 ME was a huge step forward for the franchise, and that 13.2 erased almost all of that progress, which bums me out.

I like to hope that despite my criticism of the latest iteration, I'm not a moron, so I'm not going to, nor did i suggest that the tests I run are somehow more thorough than SI's. But I also don't think that Si is running the same tests as me. SI is trying to get fun gameplay and realistic results. I am trying to figure out inefficiencies in the game that might help me win, like "pace is more important to winning than strength", or, "arrives late in the box" is better than "gets forward whenever possible".

Honestly, I'm not sure why Si would run those types of tests. Do they care which PPM is better? or do they care that the PPM's accurately reflect a players tendencies? If I discovered that pace is more important than strength, would that mean SI did something wrong? No, it would mean that pace is more important than strength, which I think is probably also true in real life.

As for how i reached these conclusions, i'll give an example: Match prep: Take a random team, about to play a home game, and have them work on "tactics only" all week to establish a baseline. sim the game. record the margin of victory (+2 or -1, etc). Reload the game, and do this for the same match 20 times. Discard any result in which a red card was given. Discard the highest and lowest outliers. Now change to attacking movement match prep. Do the same thing. Record the margin of victory. Now go to an away fixture, do the same thing. Now do the same thing with a team in a different league. Now do the same thing with a team in a different division. Now do the same thing with "defensive positioning" match prep. You get the idea. With this test, I think I used parma, chelsea, and furth.

There might be things that skew those results, like a team that uses a really defensive tactic, or a team with really good strikers, but I bet that the result is pretty good. Especially this one. The result was so emphatic that I feel pretty good about it.

But you raise an important question. Before putting out the 13.2 ME, did SI test the match engine to see if they had made creativity and flair irrelevant?

Not sure. Would they test the new ME against each specific stat to see what effect it had on results? Maybe, but i doubt it. I think they tested it thousands of times, looking for the behaviors they wanted to eliminate, and they eliminated them. But you play the game, do YOU think that through balls, creativity and flair are still incorporated as they should be? If so, you're having a different experience than me and seemly most other people on the forum.

[Newmanager]

I love this stuff. hats off to you for doing some fairly comprehensive simming and testing. There's definitely something fishy going on with 13.2.1. And while you may not have nailed the root cause on the head - the fact the weighting and importance of certain stats or settings has changed so much, implies that something is definitely not working as intended

[GodCubed]

No I don't remotely think that the FM 09 engine is better than this one, and I feel pretty sure I didn't write that. What I do think was that the 13.1 ME was a huge step forward for the franchise, and that 13.2 erased almost all of that progress, which bums me out.

Right, that's why I asked the question. You didn't SAY as such, but it seemed like you were insinuating it.

Now, re SI taking a step backwards. From looking round their forums, talking with a couple of testers and so on and so forth, I'm led to believe this latest patch was a very "things are going to get worse before they get better" kind of deal. SI have to remodel a few very complicated systems and match engine bits and pieces that were flawed (such as marking, for example). They're in the process of it and haven't finished yet, but given the huge outcry about pretty much anything with their fanbase, they've released the patch to fix the few things that were easily fixable (such as the ridiculously powerful dribbling through the centre of the pitch prior to the hotfix), but it also includes the stuff they're still working on and testing, with the end result that a few things are pretty badly off.

I like to hope that despite my criticism of the latest iteration, I'm not a moron, so I'm not going to, nor did i suggest that the tests I run are somehow more thorough than SI's. But I also don't think that Si is running the same tests as me. SI is trying to get fun gameplay and realistic results. I am trying to figure out inefficiencies in the game that might help me win, like "pace is more important to winning than strength", or, "arrives late in the box" is better than "gets forward whenever possible".

Honestly, I'm not sure why Si would run those types of tests. Do they care which PPM is better? or do they care that the PPM's accurately reflect a players tendencies? If I discovered that pace is more important than strength, would that mean SI did something wrong? No, it would mean that pace is more important than strength, which I think is probably also true in real life.

With all due respect, I'm pretty certain SI's testing also include inefficiencies in the game, particularly given that they employ people specifically to weed them out.

So what's the criticism of SI if their game and PPMs reflects real life? They're trying to make a fun game, yes, but first and foremost they're trying to make a REALISTIC simulator of a game. Though the point you bring up about PPMs is rather interesting, given that I and a couple of others feel that PPMs are given far little little attention.

As for how i reached these conclusions, i'll give an example: Match prep: Take a random team, about to play a home game, and have them work on "tactics only" all week to establish a baseline. sim the game. record the margin of victory (+2 or -1, etc). Reload the game, and do this for the same match 20 times. Discard any result in which a red card was given. Discard the highest and lowest outliers. Now change to attacking movement match prep. Do the same thing. Record the margin of victory. Now go to an away fixture, do the same thing. Now do the same thing with a team in a different league. Now do the same thing with a team in a different division. Now do the same thing with "defensive positioning" match prep. You get the idea. With this test, I think I used parma, chelsea, and furth.

There might be things that skew those results, like a team that uses a really defensive tactic, or a team with really good strikers, but I bet that the result is pretty good. Especially this one. The result was so emphatic that I feel pretty good about it.

That's impressively done, but you don't have access to the same tools SI do. They can literally destroy any kind of outliers in order to view exactly what they want to see. They need to in order to fix bugs and so on and so forth. However you do it, it will never, ever be as accurate as SI's soak tests. Either your results can be disregarded, or SI already know the faults and are working to fix them.

But you raise an important question. Before putting out the 13.2 ME, did SI test the match engine to see if they had made creativity and flair irrelevant?

Not sure. Would they test the new ME against each specific stat to see what effect it had on results? Maybe, but i doubt it. I think they tested it thousands of times, looking for the behaviors they wanted to eliminate, and they eliminated them. But you play the game, do YOU think that through balls, creativity and flair are still incorporated as they should be? If so, you're having a different experience than me and seemly most other people on the forum.

I'm not sure I'm liking this questioning of SI's quality control. They're not amateurs. They've done this for a long time, and now they have more people to check and fix. If there's a major fault with the game, they'll know about it and how to fix it. Would they test the new ME against each specific attribute? Yes. That's what they do. Go on the SI forums, talk to people who actually make the ME. This is what the testers are employed for. I cannot stress this enough.

I've got to say, I haven't tested the game as thoroughly as you have, and my answer has to be that yes, I've noticed a correlation between better attributes and better results. Maybe I'm wrong, but I've talked to people who actually help make the ME, and they've never mentioned it, even when I've raised criticisms about it.

I'll round off by pointing out that yeah, there's a lot of people on this forum who are complaining, but A. It happens literally every year. Every time a new FM comes out, people proclaim it worst than the last one and the death of the series. And B. the people who enjoy the game and don't see as many problems with it don't complain on the forums, obviously. People don't go out of their way to praise, they go out of their way to criticise. This creates a huge perceived imbalance.

Apologies for my shortness earlier. Said criticisms on this forum of the game are often badly-researched, massively unfair or just plain wrong, and given by people who are totally unwilling to back down. You're not one of them.

[spanky123]

No I don't remotely think that the FM 09 engine is better than this one, and I feel pretty sure I didn't write that. What I do think was that the 13.1 ME was a huge step forward for the franchise, and that 13.2 erased almost all of that progress, which bums me out.

I like to hope that despite my criticism of the latest iteration, I'm not a moron, so I'm not going to, nor did i suggest that the tests I run are somehow more thorough than SI's. But I also don't think that Si is running the same tests as me. SI is trying to get fun gameplay and realistic results. I am trying to figure out inefficiencies in the game that might help me win, like "pace is more important to winning than strength", or, "arrives late in the box" is better than "gets forward whenever possible".

Honestly, I'm not sure why Si would run those types of tests. Do they care which PPM is better? or do they care that the PPM's accurately reflect a players tendencies? If I discovered that pace is more important than strength, would that mean SI did something wrong? No, it would mean that pace is more important than strength, which I think is probably also true in real life.

As for how i reached these conclusions, i'll give an example: Match prep: Take a random team, about to play a home game, and have them work on "tactics only" all week to establish a baseline. sim the game. record the margin of victory (+2 or -1, etc). Reload the game, and do this for the same match 20 times. Discard any result in which a red card was given. Discard the highest and lowest outliers. Now change to attacking movement match prep. Do the same thing. Record the margin of victory. Now go to an away fixture, do the same thing. Now do the same thing with a team in a different league. Now do the same thing with a team in a different division. Now do the same thing with "defensive positioning" match prep. You get the idea. With this test, I think I used parma, chelsea, and furth.

There might be things that skew those results, like a team that uses a really defensive tactic, or a team with really good strikers, but I bet that the result is pretty good. Especially this one. The result was so emphatic that I feel pretty good about it.

But you raise an important question. Before putting out the 13.2 ME, did SI test the match engine to see if they had made creativity and flair irrelevant?

Not sure. Would they test the new ME against each specific stat to see what effect it had on results? Maybe, but i doubt it. I think they tested it thousands of times, looking for the behaviors they wanted to eliminate, and they eliminated them. But you play the game, do YOU think that through balls, creativity and flair are still incorporated as they should be? If so, you're having a different experience than me and seemly most other people on the forum.

I agree with you 100% on this and your very brave to put this on here will have all the mods ganging up on you now like most times people slag the game off

[Mike.]

I agree with you 100% on this and your very brave to put this on here will have all the mods ganging up on you now like most times people slag the game off

His post is actually useful, although there parts I disagree with and some I very much agreed. Yours however is nonsense. Newsflash, Mods have their own opinions on the game, and they are not even the same among mods. There is a difference between engaging with well thought out posts like his above which we do and dismissing mindless rants.
We never dismiss anything like the above. Its brilliantly well reasoned and well thought out
If you dont like having your opinion challenged and debated, then what are you doing on a forum?

[Mike.]

Probably one of the best posts I've seen in a while.
Simeon, its very much a case of 2 steps forward one step back. For example they know that through balls are two low in game, although they were too high in FM12.

Its interesting that you think PPMs have had too much attention though, like Godcubed said above I think they have had too little attention. The players should have more individuality. Wayne Rooney might have the same job has Kagawa behind the striker, you wouldn't expect them to play it the same irl, and the match engine should emulate it.

Regarding the ME, in any moment there are core decision processes ( for example, a striker picks up the ball outside the area, he can choose to dribble, shoot or pass) and the attributes very much influence this. The number of these moments in any one game are ridiculously high (watch a real football match, and try to follow what each play is trying to do at any one time). The issue of course is getting this underlying balance of decision making right, which is easier said than done. we've gone from too much dribbling, to too many longshots, and its not a case of trying to remove/nerf one attribute ( attributes are very much not the problem here)

[Mike.]

Q: Do players take fewer through balls, as seems obvious?
A: I can't test this directly, as I don't have a game running 13.1 anymore, but i can say this: in FM 09-12 the BEST PPM, in term of match rating and win probabilty, was, by a large margin "Tries Killer ball often". Nothing else was close. In ME 13.2, Tries Killer balls often has NO MEASURABLE EFFECT. Through balls simply don't happen enough, or aren't smart enough, to mean anything any more.

There are definitely too few through balls. However in FM12 there were too many through balls ( to ridiculously devastating effect), In real life, through balls, and accurate ones at that, are not that common, unless you are Totti. The key is the decision making behind making a through ball, and that is where the balance is off right now. Doesn't mean its impossible right now, but harder than it should be.

[T1LL13]

I agree with what Mike and GodCubed said, its more like one step back to take 3 forward. I remember FM09 being the worst game of the lot, now thats not through any testing, thats just my personal opinion. I played it, then stopped and went back to FM08. SI are doing the same thing to FM13 that they did to FM09 with regards to taking one step back to go forward, some people don't like it but in the long run it will be worth it. FM09, despite my dislike for it, gave us three of the best FM/CM games ever in 10/11 & 12. So if it turns out that SI are going back to go forward then to be honest, i can deal with the slight defects it currently has because the future results could see the greatest games we or I have ever seen or played!!