We tried Windows Copilot: What can it do and how accurately?

Microsoft made radical changes to the operating system with the Windows 11 22H2 (KB5030310) update, released on September 26. Although not all features are active for everyone yet and will be available next month, we tried Windows Copilot for you. So what are the capabilities of Windows Copilot?

Windows Copilot: An assistant you need to teach the job

First of all, we need to point out that Windows Copilot is a preview version and its testing is ongoing. Therefore, over time, this tool will become more complete, just like Bing Chat. However, in its current form it has some obvious shortcomings. Before touching on these, let’s give some information about Copilot.

We can say that Copilot is Microsoft’s Bing Chat artificial intelligence integrated into Windows. So, thanks to Microsoft’s OpenAI partnership, Copilot uses the most advanced GPT-4 model. Additionally, the image generator DALL-E 3 is integrated into the system.

So what can he do? It can actually do quite a lot. According to him, he can search the web, news or images. Copilot can also create creative and innovative content, poems, stories, codes, essays, songs, celebrity parodies and voice chat, and more. Bing Chat could already do these. The prominent difference is Windows integration. This integration works fine on its own, but the current preview version needs to be able to do more.

Can open applications and make simple Windows settings

It can also help you improve or optimize an article you have written differently. More importantly, it works integrated with Windows. For example, it can perform operations such as adjusting the volume, turning Bluetooth on and off. In addition, it can also open applications or games installed on your system, especially Windows applications. However, to do all this, just writing is not enough. After you enter your request, a dialog window appears in the conversation and you are asked to confirm.

In its current state, Copilot cannot do anything other than open applications. More precisely, unlike Bing Chat, it cannot do much. For example, “Download the Instagram app from Microsoft Store.” When I enter the prompt it only opens the Store. However, it does not initiate a process such as downloading. For example, it cannot fulfill a request such as “Shut down Windows at this time”.

Can create detailed visuals

Just like Bing, Copilot comes in three versions: Creative, Balanced, and Responsive.

Creative mode; more original and creative answers, Balanced mode; informative and useful chats, Responsive mode; It enables you to have short and understandable conversations. Let’s note that the most error-prone mode is the ‘Creative’ mode. I also used this mode to see the shortcomings. I compared the same prompts in other modes, but generally, if a prompt is answered incorrectly in one mode, the situation does not change in the others. It’s just wrong, it’s wrong in another way. On the visual production side, he has a very strong hand. For the images above, I made a very simple prompt: “Village photo taken with a Kodak camera”

You can write code or detect errors

I asked Copilot to create the relevant code for a calculator. Copilot created the necessary code using Python. Then, in the subtraction section of this code, I replaced the “=” sign with “:” and asked Copilot to detect the faulty part. He successfully explained where the error was and what caused it. Meanwhile, Microsoft stated in the codes created by Copilot that “This code was created by artificial intelligence. Examine and use carefully.” adds the warning. So you should not expect that the vehicle will always work correctly.

They have something to learn

Like most AI tools, Windows Copilot suffers from the same main, fundamental shortcoming: hallucination. For example, “When is Beşiktaş’s next match and on which channel?” I requested. He said that the next match is the UEFA Conference League match against Lugano on October 5, which is correct. But this is not the next match. Beşiktaş will face Konyaspor on October 1.

When I pointed out his mistake, he corrected his mistake by saying that his next match would be against Konyaspor on October 1. But even here there are mistakes. Copilot says that the last match between Beşiktaş and Konyaspor was on June 6, 2023 and Beşiktaş won this match 4-0.

First of all, the last encounter was on June 7, not June 6. Secondly, the match ended 3-3, not 4-0. Bugs exist in other modes as well. In the same query, Balanced mode says that Beşiktaş’s next match will be against Lugano on October 5 and that this match will be broadcast live on Star TV and NTV.

As a result, Windows Copilot is not yet that “revolutionary” feature. It’s just a much faster way to access Bing Chat. It even has some minuses than Bing. For example, a maximum of 2000 characters are offered in a single prompt. However, in Bing Chat, this is 4000. A “history” feature for chat sessions is also missing in Copilot. It is not possible to access what is said in a session when you clear the chat or start a new chat. Despite all this, we can say that Windows Copilot is the most exciting Windows feature in recent years. When we look at the possibilities ahead, we can see a lot of things Microsoft can do.