Is anyone using onprem open weigh models for programming? If so what models, gpus and feedback?
Thanks for sharing, do we know if is there any organization or production system really using it?
Best way to officially support shadow AI at government, sounds great idea
Yes but does it's not clear for me. It could be just a private interconnect to openai infrastructure. My guess is that is not hosted in gcp... That's why I'm asking
Understood yeap. Yes exactly is terraform/opentofu. The point is that we cannot do it at proxy level since we do allow any kind of operation attempt (we just filter that it's an infra req), then we generate the code…
To be precise the thing provide permission to do any operation, including destructive actions to the agent but we do set policies, auth controls and hitl force approval to validate the destructive or high impact actions…
Hi, thanks for the detailed answer! Yes it's pretty similar to the approach we are following the only difference is that we cannot provide read permissions into the endpoint so we have some kind of compliance or auditor…
Is it really gcp hosting openai in their cloud? Afaik is a third party marketplace so it looks like you're buying openai saas via gcp but not hosted in gcp. That's key difference with azure specially for egress costs if…
[flagged]
One question from the infrastructure side (i'm running a similar scenario but for infrastucture) where any mistake could not be reversible such as a deletion of a resource. How do you handle an agent that has valid…
Is anyone using onprem open weigh models for programming? If so what models, gpus and feedback?
Thanks for sharing, do we know if is there any organization or production system really using it?
Best way to officially support shadow AI at government, sounds great idea
Yes but does it's not clear for me. It could be just a private interconnect to openai infrastructure. My guess is that is not hosted in gcp... That's why I'm asking
Understood yeap. Yes exactly is terraform/opentofu. The point is that we cannot do it at proxy level since we do allow any kind of operation attempt (we just filter that it's an infra req), then we generate the code…
To be precise the thing provide permission to do any operation, including destructive actions to the agent but we do set policies, auth controls and hitl force approval to validate the destructive or high impact actions…
Hi, thanks for the detailed answer! Yes it's pretty similar to the approach we are following the only difference is that we cannot provide read permissions into the endpoint so we have some kind of compliance or auditor…
Is it really gcp hosting openai in their cloud? Afaik is a third party marketplace so it looks like you're buying openai saas via gcp but not hosted in gcp. That's key difference with azure specially for egress costs if…
[flagged]
One question from the infrastructure side (i'm running a similar scenario but for infrastucture) where any mistake could not be reversible such as a deletion of a resource. How do you handle an agent that has valid…
[flagged]