Replies: 9 (Who?), Viewed: 2778 times.
Test Subject
Original Poster
#1 Old 10th Feb 2019 at 4:28 AM Last edited by TerranWorks : 10th Feb 2019 at 4:41 AM.
Default How could I set the value of every slider in CAS and choose face presets in one fell swoop?
I've been tinkering around with the idea of creating an application which would allow the user to input images of themselves, click go, and be able to a sim that looks like them based on the pictures. I believe I have most of the actual machine learning as it's an area I am fairly comfortable with. That being said, I am very new to modding both in the sims 4 and in general so I expect the learning curve to be steep.

My question is this: If I have a vector of a certain dimension, with each dimension corresponding to a particular CAS slider or face preset slot, how can I convert the values of this vector into changes to the sim? Put another way, what is the easiest, or at least most feasible, way for me to set the values of each slider and to choose things like skin tone, hairstyle and color, and face presets from a list of numbers?

A solution needs to be able to set the values of every slider, choose skin tone, choose from presets(in cases where the result of the preset cannot be accomplished by manipulating sliders), choose gender-body-frame, choose sim orientation in CAS(rotation arrows), choose hair and hair color, and finally(this one is the kicker), I need to be able to give the mod a vector which telling it which of these choices to make and all of this has to be possible without completely restarting the game as these vectors will be created very very quickly and waiting 4 min in between every training cycle would mean the program would be learning for literally years.

For some context, my program takes in images and learns hot to turn them into a vector corresponding to changes to the sim through sliders and face presets. A screenshot of the sim is taken, and that image is compared to the original image to give the program feedback which it can learn from.

I know the machine learning aspect is possible but I don't know how to go about linking it to the sims. Any insights are appreciated!
Forum Resident
#2 Old 10th Feb 2019 at 6:35 PM
Hi! I was working on a project similar to yours a while back.

You may want to choose categorical crossentropy as your loss function for the presets the sim is going to use.
This way, you can get the preset with the highest probability and apply that item.

There is a way to obtain slider values, but it has to be done in-game instead of in CAS. Here, have a look.

And here's some example code from the link above.

@sims4.commands.Command('cas.randomize_facial_attributes', command_type=sims4.commands.CommandType.Live)
def randomize_facial_attributes(opt_sim:OptionalSimInfoParam=None, _connection=None):
from protocolbuffers import PersistenceBlobs_pb2

sim_info = get_optional_target(opt_sim, target_type=OptionalSimInfoParam, _connection=_connection)
if sim_info is None:
return False

facial_attributes = PersistenceBlobs_pb2.BlobSimFacialCustomizationData()

for modifier in itertools.chain(facial_attributes.face_modifiers, facial_attributes.body_modifiers):
modifier.amount = random.random()

sim_info.facial_attributes = facial_attributes.SerializeToString()
return True
Test Subject
Original Poster
#3 Old 11th Feb 2019 at 4:16 AM
Thank you for your reply! Doing everything outside of the CAS is definitely going to change the way I think about this problem but being able to directly set all of the slider positions will be quite wonderful. I haven't yet had a chance to test it but I am very excited about the code you posted.

You mentioned using the categorical cross entropy, do you have experience in ML?

I plan to see what I can get done tomorrow and post again with an update. I don't expect many problems but sims modding is fairly uncharted territory for me so we'll see. Thanks again.
Forum Resident
#4 Old 11th Feb 2019 at 3:37 PM
I'm a noob at Machine Learning, haha.

A quick warning though. The slider count is different per sim. Like, one adult sim may have 60 sliders, but another may have 58. I think it's because some facial presets create new sliders. I think there's a way to extract the slider id and the slider value, but I'm not sure.
Test Subject
Original Poster
#5 Old 12th Feb 2019 at 7:39 AM
Okay, so I didn't end up having a lot of free time today, but I did find some time to sit down and play around with how I was going to take screenshots of the characters outside of the CAS. Ideally, I want to remove everything from the image except for the subject to prevent the network from getting distracted basically. I ended up making this matrix style white room using some completely white walls and tiles that i downloaded along with the hidden lights found in the build menu. It was a little tricky getting rid of the shadows, in fact, I ended up having to take the roof off and set the time of day to 4am to get it to work.

One of my next challenges is going to be figuring out how to actually take the screenshots. I've recently discovered the cinematic camera and I've learned that you can set camera points and bind them to specific keys. I was thinking that I could use the program autohotkey to interface between my program and a custom script within the sims. My network would provide autohotkey with a cheats command as a string, which it will type into the cheats console and then take a screenshot. There might be an easier way to do this but this seems pretty straight forward which has its advantages.

Let me know if I've missed anything obvious, I should have more time tomorrow to tinker, hopefully I can get to the machine learning before too long! Also, in addition to this being my first mod, these have been my first posts on the forum, let me know if I need to improve the way I'm going about this. Thanks
Forum Resident
#6 Old 12th Feb 2019 at 11:51 AM
Great work so far! That was my exact same approach. Setting up an auto hotkey program to take care of the grunt work.
Test Subject
Original Poster
#7 Old 14th Feb 2019 at 5:45 AM
So, I had a bit of a revelation today. My original plan was to give my network the ability to edit a sim take a screenshot and then compare it to the picture it was suppose to be referencing to gauge how well it did (calculate its loss based on the distance between the two images to get technical), and I was prepared to learn about Names pipes and all of that to get everything working but I now realize that was overkill. This problem of turning a picture of a person into basically a sentence or a code which tells the game what values to use for each slider is extremely similar to the problem of image captioning which is an active field of research in machine learning. After looking at common solutions to that problem, I realized that they were very similar to what I was going to do so I adapted my approach to be able to utilize existing code and that was then it hit me, I can just generate a bunch of random sims, take a screenshot, grab the values for each of their sliders and store it as a single line in a text file in the same order as the screenshots. Then my network only has to turn one of those screenshots into a sim-code and compare it to the stored code to calculate its loss.

My problem is this: You mentioned that there might be a way to extract slider Id and value does anyone know how I might do this? I also need to be able to extract and set skin tone, hair preset, hair color, and face presets if I'm going to do this justice. How might I do that? Is it worth starting another thread specifically for this question?

On another note, I'm almost always working on one project or another and I've been thinking for a while now that I would be cool if I had a website to post about some of these projects. I am currently looking into creating such a website and I plan to host the finished network on it as well as a page explaining how the thing works(if it works) in layman's terms so the casual reader might get an understanding of the goings on under the hood as it were.
Forum Resident
#8 Old 14th Feb 2019 at 6:54 AM
We should form a team for this project, I'm pretty hyped to make this work!
Anyways, on your questions... skintone should be easy. For hair and hair color, you'd need to specify which assets the game can use. So something like..

valid_hairs = [id_of_hair1color1, id_of_hair1color2...]

Then you can just create a new outfit with the hair and apply it to the sim.

Facial presets is a bit harder. The slider instances and slider values can be accessed using the function here.
A quick thing to note, Sim faces aren't just sliders, they also have Sculpts. They define the texture, shape, and limits, a body part can have. So a sim's face with, I dunno, an Asian nose will look completely different to a sim with a Latin nose, even with the same slider values.
Test Subject
Original Poster
#9 Old 15th Feb 2019 at 8:13 AM
Sure I think a team makes sense as I have plenty of mod ambition but very little mod ability atm.
Test Subject
Original Poster
#10 Old 4th Mar 2019 at 6:38 PM
Hey, I wanted to make an update when I was finally able to progress on this project but I thought It'd be better to make one now. School has been taking up all of my time recently especially now that a demanding project has fallen into my lap. I intend to continue working on Face2Sim as I have taken to calling it but, I expect these updates to be sporadic though I'd like to have something to report about once a week.
Back to top