EDIT: These curl commands give a big clue as to what might be going on:
$ curl https://peer-ec1.decentraland.org/lambdas//profiles/0x2f744b5f3ec8191a4cff832600bede5ffbe38b5d
{"error":"Not Found","message":"Profile not found"}
$ curl https://peer.decentraland.org/lambdas//profiles/0x2f744b5f3ec8191a4cff832600bede5ffbe38b5d
{"timestamp":1709447376220,"avatars":[{"hasClaimedName":true,"description":"Hi, I am
It may be that profile information that is considered “stale” is discarded when new information is pushed, even if that push fails.
$ curl https://peer-ec1.decentraland.org/lambdas//profiles/0x2f744b5f3ec8191a4cff832600bede5ffbe38b5d
{"error":"Not Found","message":"Profile not found"}
But it succeeds when trying just with peer:
$ curl https://peer.decentraland.org/lambdas//profiles/0x2f744b5f3ec8191a4cff832600bede5ffbe38b5d
{"timestamp":1709447376220,"avatars":[{"hasClaimedName":true,"description":"Hi, I am psyriiis [...]
Yes, that’s ok and normal.
I believe this is the number of failed entities update, it’s reset at each catalyst restart and different catalysts might failed on different entities for different reasons.
Can you please move it back to Governance ?
This is not a question about “how do things work”.
This is an official investigation which is needing to be conducted. What the heck is going on with the product !?
Governance is for the DAO.
The catalyst is being done by the Foundation, and Governance proposals have no effect on the Foundation. This is a question for the Foundation. You can also ask it in Request for Information - Foundation
This is not an actual Proposal. This is an attempt at educating the DAO in how it should start stepping in and weighing in with its interests when out-of-control issues are seemingly not being addressed.
And so it is neither a Question, nor a Request for Information, sorry.
It’s not an out of control issue and it is being addressed when it actually becomes an issue.
You just need an explanation from the Foundation Catalyst team to understand why this is happening rather than trying to FUD by making this bigger than it actually is.
This one may be not out of control.
But it has still lasted far longer than it should have. And that is out of control.
And it is in the midst of a flurry of other issues.
And all that is far beyond out of control !
As it is no doubt seriously hurting the price of MANA.
The heart of the matter is that I, as a “user” of DCL, am suffering in a multitude of ways from the ongoing issues. From droves of potential marketplace customers being driven away, to not being able to stay in an artistic trance when creating a scene due to the malfunctioning SDK, all the way to not even being able to enjoy hanging out with the friends I made in the space due to half the room being invisible for the other half for about a week, it’s all been rather frustrating !
And from the grapevine, I can tell that I am far far far from being the only one feeling thusly.
There really is no mystery as to why DCL is a ghost town at the moment.
But there really isn’t that much to do for the main issues to be fixed.
Apologies if this thread sounded like a Question. I am still trying to figure out where and how to best amalgamate everybody’s disheveled concerns !
HP Really this last sentence is uncalled for. This went from having a constructive conversation to accusing (rather than trying to FUD) when this man is trying to help with a problem that has been happening again more and more. I actually spent no less than 6 hrs yesterday helping to test this. A real leader does not promote this kind of behavior. If there is a better way he could address this then simply let him know.
Ozy, there is a place to report bugs and they love when you send this kind of information. If you have reported it and nothing is being done I believe you can address the foundation. I would try discord also. I really appreciate what you are doing and for taking time to test it as it has been a huge problem for those of us who actually spend a lot of time here it is irritating along with the rave glitch backpack - emote glitch. If it were not a problem you, meangreendoge, metawolf, bitfiend, and others who came through wouldn’t have waisted their time. The last dollhouse event I could not see Tangy the whole time no matter how many times I refreshed. When you are unable to see the host or half the people the experience isn’t the same and I could not see if i was dancing on others or what was going on.
HPrivakos, I am not trying to call you out but rather make aware the toxic behaviors that are happening and to let you know this effects us all. We are on the same team wanting Decentraland to succeed. The success will only happen if we can appreciate each other and what we are all trying to do. Thank you
I’d like to clarify some misunderstandings. The Failed field in the Catalyst monitor within the Content Server indicates the number of entities that attempted to deploy to a server at a specific pointer (wallet, parcel, urn, etc) but failed. The reasons for the failure may include mismatched metadata, insufficient permissions for the wallet, or an invalid signature (you may check each node’s datils with the URL https://[node-host]/content/failed-deployments). The failure count can vary between nodes. Because the nodes are decentralized, they receive and synchronize content with the rest of the network, but they do not propagate entities that fail to deploy. For instance, nodes like Loki that have not updated their content server version for some time may fail to deploy entities because they are not running the latest validations.
The Catalyst monitor is widely used, and node operators rely on it extensively. In addition to using the monitor, operators set up alerts for various issues, such as node failures, CPU or memory spikes, high error rates, service downtime, and more.
On the other hand, there is an ongoing issue being investigated about players not being able to see each other and you may check the status on the status page https://status.decentraland.org/
and the support channels.
So I had done that, and was preparing to comment on that next, as one of the results from https://peer-ec1.decentraland.org/content/failed-deployments is for 0x2f744b5f3ec8191a4cff832600bede5ffbe38b5d.
And as stated above, I get these curl results, showing the profile information missing on the load-balancer:
$ curl https://peer-ec1.decentraland.org/lambdas//profiles/0x2f744b5f3ec8191a4cff832600bede5ffbe38b5d
{"error":"Not Found","message":"Profile not found"}
$ curl https://peer.decentraland.org/lambdas//profiles/0x2f744b5f3ec8191a4cff832600bede5ffbe38b5d
{"timestamp":1709447376220,"avatars":[{"hasClaimedName":true,"description":"Hi, I am
And I assumed that the results from content/failed-deployments meant that this profile information wasn’t properly propagated to the load-balancers.
But the timestamps actually contradict this…
The profile info available has a timestamp of 1709..., but all failures on the load balancers show a timestamp of at least 1711....
It seemed like new profile information wasn’t being properly propagated ! And that no check was in place for catch stuff like this. Which would have been a Governance issue.
But in the new light of these timestamps, it does indeed make more sense for this to be Request for Information - Foundation - Decentraland. Unfortunately the category seems to be closed at the moment…
The timestamps from content/failed-deployments and the curl results suggest that the heart of the issue may be: failed deploys somehow causing existing profile information to be discarded from the relevant catalysts.
It seems there is another misunderstanding. The load balancer at peer.decentraland.org sits in front of the nodes, but the nodes do not propagate content to a load balancer. The load balancer simply directs requests for information to the nodes on the network, and it is the nodes themselves that handle content synchronization. The profile you mentioned is present in peer-eu1, the profile timestamp is old (5 months ago) and hasn’t been updated in a long time, making it stale content. Nodes with Garbage Collection enabled will remove stale content to conserve disk space, but peer-eu1 has Garbage Collection turned off, so it retains all content.
It feels like you may be framing this as a bug or a larger issue than it is, rather than seeking assistance to resolve the issue you’re facing. It would be more helpful if you could outline your current needs and limitations so we can offer appropriate support. Additionally, there is a Catalyst Discord channel where we can communicate more efficiently.
