I am still quite new at F# and trying to figure out how to make my own kind of type that can hold any number of "A"'s before if should how a value in the end.

As an example it could be like:

A(A(A(A(A(0))))).

if I try to make the type like this I try to declare it like this:

type test =
          | A of int
          | A of test;;

It tells me that I can't declare the same type twice as I have duplicates. Is there any way to work around this or do I really need to make the last node another name like this:

type test =
          | B of int
          | A of test;;

and the result would then be:

A(A(A(A(B(0)))))

Any help please?

I'm not sure if this will fit your other limitations (which you're not telling us), but this can be easily done by making the type generic:

type test<'a> = A of 'a

let a0 = A 0
let a1 = A(A(0))
let a5 = A(A(A(A(A(0)))))

This approach has this trait that values with different number of As have different types - i.e. in the above snippet, a0 has type test<int>, but a1 has type test<test<int>>. Whether this is an advantage or a disadvantage depends on your larger context.

That being said, I find myself wondering why you would want to do this in the first place, except as an abstract exercise in language syntax. Perhaps if you clarify your root problem and/or domain, the community would be able to help you better.